2024-02-18T00:57:58.528 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-02-18T00:57:58.528 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039 --description rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} --owner scheduled_yuriw@teuthology --verbose -- /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/orig.config.yaml 2024-02-18T00:57:58.559 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T00:57:58.734 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039 branch: pacific-release description: rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} email: yweinste@redhat.com first_in_suite: false job_id: '7565039' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '18.04' overrides: admin_socket: branch: pacific-release ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms bind msgr2: false ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(MON_DOWN\) - missing hit_sets - do not have an application enabled - application not enabled on pool - pool application - mons down - out of quorum - needs hit_set_type to be set but it is not mon_bind_msgr2: false sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 extra_packages: - cephadm workunit: branch: pacific-release sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 owner: scheduled_yuriw@teuthology priority: 51 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 264 sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: pacific-release suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 targets: smithi105.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE= smithi155.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-02-17_15:55:51 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-02-18T00:57:58.734 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa; will attempt to use it 2024-02-18T00:57:58.735 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_81bd20d634209c7cb82c18be12b4b5a05643ebf1/qa/tasks 2024-02-18T00:57:58.735 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-02-18T00:57:58.737 INFO:teuthology.task.internal:Checking packages... 2024-02-18T00:57:58.761 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '81bd20d634209c7cb82c18be12b4b5a05643ebf1' 2024-02-18T00:57:58.762 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-02-18T00:57:58.762 INFO:teuthology.packaging:ref: None 2024-02-18T00:57:58.762 INFO:teuthology.packaging:tag: None 2024-02-18T00:57:58.762 INFO:teuthology.packaging:branch: pacific-release 2024-02-18T00:57:58.762 INFO:teuthology.packaging:sha1: 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T00:57:58.762 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&ref=pacific-release 2024-02-18T00:57:58.903 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-548-g81bd20d6-1bionic 2024-02-18T00:57:58.905 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-02-18T00:57:58.912 INFO:teuthology.task.internal:no buildpackages task found 2024-02-18T00:57:58.912 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-02-18T00:57:58.933 INFO:teuthology.task.internal:Saving configuration 2024-02-18T00:57:58.944 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-02-18T00:57:58.950 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T00:57:58.976 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:52:23.760472', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII4U6ItCfOdoN1WdROC3KO40lcw/kzXeIM8M8q950kfL'} 2024-02-18T00:57:58.999 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:52:23.758881', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAII4U6ItCfOdoN1WdROC3KO40lcw/kzXeIM8M8q950kfL'} 2024-02-18T00:57:58.999 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-02-18T00:57:59.005 INFO:teuthology.task.internal:roles: ubuntu@smithi105.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-02-18T00:57:59.006 INFO:teuthology.task.internal:roles: ubuntu@smithi155.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-02-18T00:57:59.006 INFO:teuthology.run_tasks:Running task console_log... 2024-02-18T00:57:59.123 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f15adf2c0d0>, signals=[15]) 2024-02-18T00:57:59.124 INFO:teuthology.run_tasks:Running task internal.connect... 2024-02-18T00:57:59.130 INFO:teuthology.task.internal:Opening connections... 2024-02-18T00:57:59.130 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi105.front.sepia.ceph.com 2024-02-18T00:57:59.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:57:59.212 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi155.front.sepia.ceph.com 2024-02-18T00:57:59.213 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:57:59.291 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-02-18T00:57:59.298 DEBUG:teuthology.orchestra.run.smithi105:> uname -m 2024-02-18T00:57:59.331 INFO:teuthology.orchestra.run.smithi105.stdout:x86_64 2024-02-18T00:57:59.331 DEBUG:teuthology.orchestra.run.smithi105:> cat /etc/os-release 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:NAME="Ubuntu" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:ID=ubuntu 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:ID_LIKE=debian 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_ID="18.04" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:VERSION_CODENAME=bionic 2024-02-18T00:57:59.383 INFO:teuthology.orchestra.run.smithi105.stdout:UBUNTU_CODENAME=bionic 2024-02-18T00:57:59.384 INFO:teuthology.lock.ops:Updating smithi105.front.sepia.ceph.com on lock server 2024-02-18T00:57:59.408 DEBUG:teuthology.orchestra.run.smithi155:> uname -m 2024-02-18T00:57:59.414 INFO:teuthology.orchestra.run.smithi155.stdout:x86_64 2024-02-18T00:57:59.415 DEBUG:teuthology.orchestra.run.smithi155:> cat /etc/os-release 2024-02-18T00:57:59.464 INFO:teuthology.orchestra.run.smithi155.stdout:NAME="Ubuntu" 2024-02-18T00:57:59.464 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION="18.04.6 LTS (Bionic Beaver)" 2024-02-18T00:57:59.464 INFO:teuthology.orchestra.run.smithi155.stdout:ID=ubuntu 2024-02-18T00:57:59.464 INFO:teuthology.orchestra.run.smithi155.stdout:ID_LIKE=debian 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:PRETTY_NAME="Ubuntu 18.04.6 LTS" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_ID="18.04" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:HOME_URL="https://www.ubuntu.com/" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:VERSION_CODENAME=bionic 2024-02-18T00:57:59.465 INFO:teuthology.orchestra.run.smithi155.stdout:UBUNTU_CODENAME=bionic 2024-02-18T00:57:59.465 INFO:teuthology.lock.ops:Updating smithi155.front.sepia.ceph.com on lock server 2024-02-18T00:57:59.492 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-02-18T00:57:59.504 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-02-18T00:57:59.513 INFO:teuthology.task.internal:Checking for old test directory... 2024-02-18T00:57:59.513 DEBUG:teuthology.orchestra.run.smithi105:> test '!' -e /home/ubuntu/cephtest 2024-02-18T00:57:59.515 DEBUG:teuthology.orchestra.run.smithi155:> test '!' -e /home/ubuntu/cephtest 2024-02-18T00:57:59.521 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-02-18T00:57:59.527 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-02-18T00:57:59.527 DEBUG:teuthology.orchestra.run.smithi105:> test -z $(ls -A /var/lib/ceph) 2024-02-18T00:57:59.562 DEBUG:teuthology.orchestra.run.smithi155:> test -z $(ls -A /var/lib/ceph) 2024-02-18T00:57:59.635 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-02-18T00:57:59.694 INFO:teuthology.run_tasks:Running task kernel... 2024-02-18T00:57:59.709 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-02-18T00:57:59.709 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-02-18T00:57:59.710 DEBUG:teuthology.orchestra.run.smithi105:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T00:57:59.710 DEBUG:teuthology.orchestra.run.smithi155:> test -f /run/.containerenv -o -f /.dockerenv 2024-02-18T00:57:59.715 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:57:59.715 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-02-18T00:57:59.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T00:57:59.716 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-02-18T00:57:59.762 INFO:teuthology.orchestra.run.smithi105.stdout:4.15.0-200-generic 2024-02-18T00:57:59.762 INFO:teuthology.task.kernel:Running kernel on smithi105: 4.15.0-200-generic 2024-02-18T00:57:59.763 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get clean 2024-02-18T00:57:59.764 INFO:teuthology.orchestra.run.smithi155.stdout:4.15.0-200-generic 2024-02-18T00:57:59.764 INFO:teuthology.task.kernel:Running kernel on smithi155: 4.15.0-200-generic 2024-02-18T00:57:59.764 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-02-18T00:57:59.871 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-02-18T00:57:59.872 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-02-18T00:58:00.140 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2024-02-18T00:58:00.157 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T00:58:00.174 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2024-02-18T00:58:00.254 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T00:58:00.255 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB] 2024-02-18T00:58:00.313 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB] 2024-02-18T00:58:00.320 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB] 2024-02-18T00:58:00.532 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,379 kB] 2024-02-18T00:58:00.651 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [83.3 kB] 2024-02-18T00:58:00.667 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,045 kB] 2024-02-18T00:58:00.712 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,717 kB] 2024-02-18T00:58:00.741 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [2,717 kB] 2024-02-18T00:58:00.826 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [467 kB] 2024-02-18T00:58:00.840 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1,317 kB] 2024-02-18T00:58:00.842 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,665 kB] 2024-02-18T00:58:00.933 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [33.0 kB] 2024-02-18T00:58:00.933 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [182 kB] 2024-02-18T00:58:00.937 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [1,078 kB] 2024-02-18T00:58:00.960 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,303 kB] 2024-02-18T00:58:00.979 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [308 kB] 2024-02-18T00:58:00.992 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,008 B] 2024-02-18T00:58:00.992 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.8 kB] 2024-02-18T00:58:00.992 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [3,928 B] 2024-02-18T00:58:01.017 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/main i386 Packages [1,665 kB] 2024-02-18T00:58:01.076 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [554 kB] 2024-02-18T00:58:01.093 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [39.7 kB] 2024-02-18T00:58:01.096 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1,347 kB] 2024-02-18T00:58:01.202 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [187 kB] 2024-02-18T00:58:01.206 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,663 kB] 2024-02-18T00:58:01.234 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,915 kB] 2024-02-18T00:58:01.248 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://security.ubuntu.com/ubuntu bionic-security/main i386 Packages [1,379 kB] 2024-02-18T00:58:01.278 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://security.ubuntu.com/ubuntu bionic-security/main Translation-en [467 kB] 2024-02-18T00:58:01.283 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [421 kB] 2024-02-18T00:58:01.286 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://security.ubuntu.com/ubuntu bionic-security/restricted i386 Packages [33.0 kB] 2024-02-18T00:58:01.288 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [1,317 kB] 2024-02-18T00:58:01.288 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [3,045 kB] 2024-02-18T00:58:01.292 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [25.6 kB] 2024-02-18T00:58:01.293 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.2 kB] 2024-02-18T00:58:01.293 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,088 B] 2024-02-18T00:58:01.294 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic-backports/main i386 Packages [53.2 kB] 2024-02-18T00:58:01.295 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [53.3 kB] 2024-02-18T00:58:01.296 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [18.1 kB] 2024-02-18T00:58:01.312 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [18.2 kB] 2024-02-18T00:58:01.338 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://security.ubuntu.com/ubuntu bionic-security/restricted Translation-en [182 kB] 2024-02-18T00:58:01.342 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://security.ubuntu.com/ubuntu bionic-security/universe i386 Packages [1,078 kB] 2024-02-18T00:58:01.365 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [1,303 kB] 2024-02-18T00:58:01.391 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://security.ubuntu.com/ubuntu bionic-security/universe Translation-en [308 kB] 2024-02-18T00:58:01.419 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main Translation-en [554 kB] 2024-02-18T00:58:01.420 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [19.8 kB] 2024-02-18T00:58:01.435 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic-updates/restricted i386 Packages [39.7 kB] 2024-02-18T00:58:01.435 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [1,347 kB] 2024-02-18T00:58:01.465 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic-updates/restricted Translation-en [187 kB] 2024-02-18T00:58:01.469 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/universe i386 Packages [1,663 kB] 2024-02-18T00:58:01.519 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1,915 kB] 2024-02-18T00:58:01.584 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://security.ubuntu.com/ubuntu bionic-security/multiverse i386 Packages [6,008 B] 2024-02-18T00:58:01.665 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://security.ubuntu.com/ubuntu bionic-security/multiverse Translation-en [3,928 B] 2024-02-18T00:58:01.742 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe Translation-en [421 kB] 2024-02-18T00:58:02.268 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse i386 Packages [11.2 kB] 2024-02-18T00:58:02.269 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [25.6 kB] 2024-02-18T00:58:02.272 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse Translation-en [6,088 B] 2024-02-18T00:58:02.273 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [53.3 kB] 2024-02-18T00:58:02.280 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/main i386 Packages [53.2 kB] 2024-02-18T00:58:02.288 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [18.2 kB] 2024-02-18T00:58:02.291 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/universe i386 Packages [18.1 kB] 2024-02-18T00:58:03.526 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 20.1 MB in 3s (5,941 kB/s) 2024-02-18T00:58:03.981 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 20.1 MB in 4s (5,214 kB/s) 2024-02-18T00:58:05.205 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T00:58:05.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-18T00:58:05.338 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T00:58:05.598 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-02-18T00:58:05.599 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-02-18T00:58:05.734 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T00:58:05.787 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-18T00:58:05.816 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2024-02-18T00:58:05.816 INFO:teuthology.orchestra.run.smithi105.stdout: linux-generic linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-18T00:58:05.816 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-generic linux-image-4.15.0-213-generic 2024-02-18T00:58:05.816 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-4.15.0-213-generic linux-modules-extra-4.15.0-213-generic 2024-02-18T00:58:05.819 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2024-02-18T00:58:05.819 INFO:teuthology.orchestra.run.smithi105.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2024-02-18T00:58:05.819 INFO:teuthology.orchestra.run.smithi105.stdout:Recommended packages: 2024-02-18T00:58:05.819 INFO:teuthology.orchestra.run.smithi105.stdout: thermald 2024-02-18T00:58:05.874 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T00:58:05.881 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2024-02-18T00:58:05.881 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-18T00:58:05.881 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-4.15.0-213-generic linux-modules-4.15.0-213-generic 2024-02-18T00:58:05.881 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-extra-4.15.0-213-generic 2024-02-18T00:58:05.883 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages will be upgraded: 2024-02-18T00:58:05.884 INFO:teuthology.orchestra.run.smithi105.stdout: linux-generic linux-headers-generic linux-image-generic 2024-02-18T00:58:05.946 INFO:teuthology.orchestra.run.smithi105.stdout:3 upgraded, 5 newly installed, 0 to remove and 165 not upgraded. 2024-02-18T00:58:05.946 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 67.4 MB of archives. 2024-02-18T00:58:05.946 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 351 MB of additional disk space will be used. 2024-02-18T00:58:05.946 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-213-generic amd64 4.15.0-213.224 [13.4 MB] 2024-02-18T00:58:06.155 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-02-18T00:58:06.155 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-02-18T00:58:06.262 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-213-generic amd64 4.15.0-213.224 [8,111 kB] 2024-02-18T00:58:06.368 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-213-generic amd64 4.15.0-213.224 [33.7 MB] 2024-02-18T00:58:06.375 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-02-18T00:58:06.375 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-18T00:58:06.375 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-generic linux-image-4.15.0-213-generic 2024-02-18T00:58:06.375 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-4.15.0-213-generic linux-modules-extra-4.15.0-213-generic 2024-02-18T00:58:06.378 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-02-18T00:58:06.378 INFO:teuthology.orchestra.run.smithi155.stdout: fdutils linux-doc-4.15.0 | linux-source-4.15.0 linux-tools 2024-02-18T00:58:06.378 INFO:teuthology.orchestra.run.smithi155.stdout:Recommended packages: 2024-02-18T00:58:06.378 INFO:teuthology.orchestra.run.smithi155.stdout: thermald 2024-02-18T00:58:06.440 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-02-18T00:58:06.440 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-4.15.0-213 linux-headers-4.15.0-213-generic 2024-02-18T00:58:06.440 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-4.15.0-213-generic linux-modules-4.15.0-213-generic 2024-02-18T00:58:06.440 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-extra-4.15.0-213-generic 2024-02-18T00:58:06.444 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-02-18T00:58:06.444 INFO:teuthology.orchestra.run.smithi155.stdout: linux-generic linux-headers-generic linux-image-generic 2024-02-18T00:58:06.768 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.213.196 [1,864 B] 2024-02-18T00:58:06.769 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.213.196 [2,460 B] 2024-02-18T00:58:06.769 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213 all 4.15.0-213.224 [11.0 MB] 2024-02-18T00:58:06.898 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213-generic amd64 4.15.0-213.224 [1,250 kB] 2024-02-18T00:58:06.912 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.213.196 [2,360 B] 2024-02-18T00:58:07.651 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 67.4 MB in 1s (66.7 MB/s) 2024-02-18T00:58:07.686 INFO:teuthology.orchestra.run.smithi155.stdout:3 upgraded, 5 newly installed, 0 to remove and 165 not upgraded. 2024-02-18T00:58:07.686 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 67.4 MB of archives. 2024-02-18T00:58:07.686 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 351 MB of additional disk space will be used. 2024-02-18T00:58:07.686 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-4.15.0-213-generic amd64 4.15.0-213.224 [13.4 MB] 2024-02-18T00:58:07.852 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-modules-4.15.0-213-generic. 2024-02-18T00:58:08.019 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-4.15.0-213-generic amd64 4.15.0-213.224 [8,111 kB] 2024-02-18T00:58:08.132 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-modules-extra-4.15.0-213-generic amd64 4.15.0-213.224 [33.7 MB] 2024-02-18T00:58:08.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-generic amd64 4.15.0.213.196 [1,864 B] 2024-02-18T00:58:08.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-image-generic amd64 4.15.0.213.196 [2,460 B] 2024-02-18T00:58:08.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213 all 4.15.0-213.224 [11.0 MB] 2024-02-18T00:58:08.675 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-4.15.0-213-generic amd64 4.15.0-213.224 [1,250 kB] 2024-02-18T00:58:08.689 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 linux-headers-generic amd64 4.15.0.213.196 [2,360 B] 2024-02-18T00:58:08.721 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 135963 files and directories currently installed.) 2024-02-18T00:58:08.724 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../0-linux-modules-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:08.798 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:09.524 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 67.4 MB in 2s (30.2 MB/s) 2024-02-18T00:58:09.743 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-4.15.0-213-generic. 2024-02-18T00:58:10.465 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-image-4.15.0-213-generic. 2024-02-18T00:58:10.481 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../1-linux-image-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:10.574 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:10.697 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 135963 files and directories currently installed.) 2024-02-18T00:58:10.701 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../0-linux-modules-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:10.789 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:11.119 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-213-generic. 2024-02-18T00:58:11.135 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:11.177 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:12.581 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-image-4.15.0-213-generic. 2024-02-18T00:58:12.597 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../1-linux-image-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:12.709 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:13.259 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-modules-extra-4.15.0-213-generic. 2024-02-18T00:58:13.274 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../2-linux-modules-extra-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:13.317 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:14.978 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../3-linux-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:15.080 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:15.390 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:15.516 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-image-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:15.853 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-headers-4.15.0-213. 2024-02-18T00:58:15.867 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../5-linux-headers-4.15.0-213_4.15.0-213.224_all.deb ... 2024-02-18T00:58:15.911 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-18T00:58:17.277 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../3-linux-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:17.411 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:17.755 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../4-linux-image-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:17.881 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-image-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:18.217 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-4.15.0-213. 2024-02-18T00:58:18.234 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../5-linux-headers-4.15.0-213_4.15.0-213.224_all.deb ... 2024-02-18T00:58:18.300 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-18T00:58:20.535 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package linux-headers-4.15.0-213-generic. 2024-02-18T00:58:20.558 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../6-linux-headers-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:20.602 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:22.930 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:23.041 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking linux-headers-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:23.148 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package linux-headers-4.15.0-213-generic. 2024-02-18T00:58:23.174 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../6-linux-headers-4.15.0-213-generic_4.15.0-213.224_amd64.deb ... 2024-02-18T00:58:23.237 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:23.347 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:23.932 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:24.420 INFO:teuthology.orchestra.run.smithi105.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:24.420 INFO:teuthology.orchestra.run.smithi105.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:24.420 INFO:teuthology.orchestra.run.smithi105.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:24.420 INFO:teuthology.orchestra.run.smithi105.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:24.534 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:25.078 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-18T00:58:25.204 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-image-generic (4.15.0.213.196) ... 2024-02-18T00:58:25.330 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:25.456 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-headers-generic (4.15.0.213.196) ... 2024-02-18T00:58:25.582 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up linux-generic (4.15.0.213.196) ... 2024-02-18T00:58:25.708 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:25.720 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../7-linux-headers-generic_4.15.0.213.196_amd64.deb ... 2024-02-18T00:58:25.865 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking linux-headers-generic (4.15.0.213.196) over (4.15.0.200.183) ... 2024-02-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi105.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-18T00:58:25.919 INFO:teuthology.orchestra.run.smithi105.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:26.226 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:26.878 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:27.370 INFO:teuthology.orchestra.run.smithi155.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:27.370 INFO:teuthology.orchestra.run.smithi155.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:27.370 INFO:teuthology.orchestra.run.smithi155.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:27.370 INFO:teuthology.orchestra.run.smithi155.stdout:I: /initrd.img is now a symlink to boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:27.505 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-modules-extra-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:28.065 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-4.15.0-213 (4.15.0-213.224) ... 2024-02-18T00:58:28.191 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-image-generic (4.15.0.213.196) ... 2024-02-18T00:58:28.317 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:28.459 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-headers-generic (4.15.0.213.196) ... 2024-02-18T00:58:28.585 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up linux-generic (4.15.0.213.196) ... 2024-02-18T00:58:28.729 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for linux-image-4.15.0-213-generic (4.15.0-213.224) ... 2024-02-18T00:58:28.935 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-02-18T00:58:28.935 INFO:teuthology.orchestra.run.smithi155.stdout:update-initramfs: Generating /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:40.911 INFO:teuthology.orchestra.run.smithi105.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-18T00:58:40.911 INFO:teuthology.orchestra.run.smithi105.stdout:Sourcing file `/etc/default/grub' 2024-02-18T00:58:40.938 INFO:teuthology.orchestra.run.smithi105.stdout:Generating grub configuration file ... 2024-02-18T00:58:41.305 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:41.355 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:41.741 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:41.775 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:41.848 INFO:teuthology.orchestra.run.smithi105.stdout:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-18T00:58:41.884 INFO:teuthology.orchestra.run.smithi105.stdout:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-18T00:58:42.864 INFO:teuthology.orchestra.run.smithi105.stdout:done 2024-02-18T00:58:43.326 DEBUG:teuthology.orchestra.run.smithi105:> dpkg -s linux-image-generic 2024-02-18T00:58:43.346 INFO:teuthology.orchestra.run.smithi105.stdout:Package: linux-image-generic 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Status: install ok installed 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Priority: optional 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Section: kernel 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Installed-Size: 18 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Maintainer: Ubuntu Kernel Team 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: amd64 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Source: linux-meta 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Version: 4.15.0.213.196 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Recommends: thermald 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout:Description: Generic Linux kernel image 2024-02-18T00:58:43.347 INFO:teuthology.orchestra.run.smithi105.stdout: This package will always depend on the latest generic kernel image 2024-02-18T00:58:43.348 INFO:teuthology.orchestra.run.smithi105.stdout: available. 2024-02-18T00:58:43.348 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-200-generic Expected: 4.15.0-213-generic 2024-02-18T00:58:43.348 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T00:58:43.348 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T00:58:43.348 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-02-18T00:58:43.348 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi105.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T00:58:43.349 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get clean 2024-02-18T00:58:43.446 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-02-18T00:58:43.571 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:1 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T00:58:43.579 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-18T00:58:43.599 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-18T00:58:43.704 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-18T00:58:44.245 INFO:teuthology.orchestra.run.smithi155.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-02-18T00:58:44.246 INFO:teuthology.orchestra.run.smithi155.stdout:Sourcing file `/etc/default/grub' 2024-02-18T00:58:44.290 INFO:teuthology.orchestra.run.smithi155.stdout:Generating grub configuration file ... 2024-02-18T00:58:44.690 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:44.738 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:45.140 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:45.164 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:45.240 INFO:teuthology.orchestra.run.smithi155.stdout:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-18T00:58:45.270 INFO:teuthology.orchestra.run.smithi155.stdout:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-18T00:58:45.664 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T00:58:45.715 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-18T00:58:45.797 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T00:58:46.079 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-02-18T00:58:46.080 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout:linux-image-generic is already the newest version (4.15.0.213.196). 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-18T00:58:46.291 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-18T00:58:46.344 INFO:teuthology.orchestra.run.smithi105.stdout:0 upgraded, 0 newly installed, 0 to remove and 165 not upgraded. 2024-02-18T00:58:46.346 DEBUG:teuthology.orchestra.run.smithi105:> dpkg -s linux-image-generic 2024-02-18T00:58:46.363 INFO:teuthology.orchestra.run.smithi105.stdout:Package: linux-image-generic 2024-02-18T00:58:46.363 INFO:teuthology.orchestra.run.smithi105.stdout:Status: install ok installed 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Priority: optional 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Section: kernel 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Installed-Size: 18 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Maintainer: Ubuntu Kernel Team 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Architecture: amd64 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Source: linux-meta 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Version: 4.15.0.213.196 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Recommends: thermald 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout:Description: Generic Linux kernel image 2024-02-18T00:58:46.364 INFO:teuthology.orchestra.run.smithi105.stdout: This package will always depend on the latest generic kernel image 2024-02-18T00:58:46.365 INFO:teuthology.orchestra.run.smithi105.stdout: available. 2024-02-18T00:58:46.365 DEBUG:teuthology.orchestra.run.smithi105:> mktemp 2024-02-18T00:58:46.410 INFO:teuthology.orchestra.run.smithi105.stdout:/tmp/tmp.GkONd3tEBG 2024-02-18T00:58:46.411 DEBUG:teuthology.orchestra.run.smithi105:> sudo cp /boot/grub/grub.cfg /tmp/tmp.GkONd3tEBG 2024-02-18T00:58:46.468 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0666 /tmp/tmp.GkONd3tEBG 2024-02-18T00:58:46.606 DEBUG:teuthology.orchestra.remote:smithi105:/tmp/tmp.GkONd3tEBG is 10KB 2024-02-18T00:58:46.617 INFO:teuthology.orchestra.run.smithi155.stdout:done 2024-02-18T00:58:46.619 DEBUG:teuthology.orchestra.run.smithi105:> rm -fr /tmp/tmp.GkONd3tEBG 2024-02-18T00:58:46.626 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-18T00:58:46.688 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T00:58:46.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-18T00:58:46.688 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-18T00:58:46.758 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-213-generic 2024-02-18T00:58:46.759 DEBUG:teuthology.orchestra.run.smithi105:> sudo update-grub 2024-02-18T00:58:47.130 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 18 2024-02-18T00:58:47.151 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 4.15.0.213.196 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-02-18T00:58:47.152 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-02-18T00:58:47.152 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.15.0-200-generic Expected: 4.15.0-213-generic 2024-02-18T00:58:47.153 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-02-18T00:58:47.153 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-02-18T00:58:47.153 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-02-18T00:58:47.153 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi155.front.sepia.ceph.com, path=None, version=distro) 2024-02-18T00:58:47.153 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get clean 2024-02-18T00:58:47.263 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-02-18T00:58:47.519 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-18T00:58:48.008 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T00:58:48.089 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-18T00:58:48.130 INFO:teuthology.orchestra.run.smithi105.stderr:Sourcing file `/etc/default/grub' 2024-02-18T00:58:48.149 INFO:teuthology.orchestra.run.smithi105.stderr:Generating grub configuration file ... 2024-02-18T00:58:48.170 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-18T00:58:48.423 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:48.435 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:48.743 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:48.755 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:48.831 INFO:teuthology.orchestra.run.smithi105.stderr:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-18T00:58:48.844 INFO:teuthology.orchestra.run.smithi105.stderr:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-18T00:58:49.445 INFO:teuthology.orchestra.run.smithi105.stderr:done 2024-02-18T00:58:49.447 DEBUG:teuthology.orchestra.run.smithi105:> sudo shutdown -r now 2024-02-18T00:58:49.980 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T00:58:50.033 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-02-18T00:58:50.115 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T00:58:50.394 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-02-18T00:58:50.394 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-02-18T00:58:50.601 INFO:teuthology.orchestra.run.smithi155.stdout:linux-image-generic is already the newest version (4.15.0.213.196). 2024-02-18T00:58:50.601 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-02-18T00:58:50.601 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-18T00:58:50.601 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-18T00:58:50.602 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-18T00:58:50.602 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-18T00:58:50.657 INFO:teuthology.orchestra.run.smithi155.stdout:0 upgraded, 0 newly installed, 0 to remove and 165 not upgraded. 2024-02-18T00:58:50.659 DEBUG:teuthology.orchestra.run.smithi155:> dpkg -s linux-image-generic 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Package: linux-image-generic 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Status: install ok installed 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Priority: optional 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Section: kernel 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Installed-Size: 18 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Maintainer: Ubuntu Kernel Team 2024-02-18T00:58:50.726 INFO:teuthology.orchestra.run.smithi155.stdout:Architecture: amd64 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Source: linux-meta 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Version: 4.15.0.213.196 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Provides: wireguard-modules (= 1.0.20201112-1~18.04.4), zfs-modules (= 0.7.5-1ubuntu16.12) 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Depends: linux-image-4.15.0-213-generic, linux-modules-extra-4.15.0-213-generic, linux-firmware, intel-microcode, amd64-microcode 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Recommends: thermald 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout:Description: Generic Linux kernel image 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout: This package will always depend on the latest generic kernel image 2024-02-18T00:58:50.727 INFO:teuthology.orchestra.run.smithi155.stdout: available. 2024-02-18T00:58:50.727 DEBUG:teuthology.orchestra.run.smithi155:> mktemp 2024-02-18T00:58:50.772 INFO:teuthology.orchestra.run.smithi155.stdout:/tmp/tmp.fWJEz5rUAl 2024-02-18T00:58:50.772 DEBUG:teuthology.orchestra.run.smithi155:> sudo cp /boot/grub/grub.cfg /tmp/tmp.fWJEz5rUAl 2024-02-18T00:58:50.825 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0666 /tmp/tmp.fWJEz5rUAl 2024-02-18T00:58:50.954 DEBUG:teuthology.orchestra.remote:smithi155:/tmp/tmp.fWJEz5rUAl is 10KB 2024-02-18T00:58:50.966 DEBUG:teuthology.orchestra.run.smithi155:> rm -fr /tmp/tmp.fWJEz5rUAl 2024-02-18T00:58:50.972 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-02-18T00:58:51.029 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T00:58:51.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-02-18T00:58:51.029 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-02-18T00:58:51.098 INFO:teuthology.task.kernel:Distro Kernel Version: 4.15.0-213-generic 2024-02-18T00:58:51.098 DEBUG:teuthology.orchestra.run.smithi155:> sudo update-grub 2024-02-18T00:58:52.625 INFO:teuthology.orchestra.run.smithi155.stderr:Sourcing file `/etc/default/grub' 2024-02-18T00:58:52.639 INFO:teuthology.orchestra.run.smithi155.stderr:Generating grub configuration file ... 2024-02-18T00:58:52.904 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-4.15.0-213-generic 2024-02-18T00:58:52.917 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-4.15.0-213-generic 2024-02-18T00:58:53.235 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-4.15.0-200-generic 2024-02-18T00:58:53.246 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-4.15.0-200-generic 2024-02-18T00:58:53.318 INFO:teuthology.orchestra.run.smithi155.stderr:Found linux image: /boot/vmlinuz-4.15.0-194-generic 2024-02-18T00:58:53.330 INFO:teuthology.orchestra.run.smithi155.stderr:Found initrd image: /boot/initrd.img-4.15.0-194-generic 2024-02-18T00:58:54.305 INFO:teuthology.orchestra.run.smithi155.stderr:done 2024-02-18T00:58:54.307 DEBUG:teuthology.orchestra.run.smithi155:> sudo shutdown -r now 2024-02-18T00:59:19.472 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-02-18T00:59:19.473 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-02-18T00:59:19.474 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:59:24.312 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-02-18T00:59:24.312 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T00:59:24.313 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T00:59:58.908 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-02-18T01:00:07.916 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T01:00:07.917 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T01:00:19.484 DEBUG:teuthology.orchestra.remote:timed out 2024-02-18T01:00:28.492 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-02-18T01:00:28.493 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T01:00:28.585 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-02-18T01:00:28.905 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-02-18T01:00:28.905 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.15.0-213-generic"... 2024-02-18T01:00:28.905 DEBUG:teuthology.orchestra.run.smithi105:> uname -r 2024-02-18T01:00:28.954 INFO:teuthology.orchestra.run.smithi105.stdout:4.15.0-213-generic 2024-02-18T01:00:28.955 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-213-generic vs 4.15.0-213-generic 2024-02-18T01:00:28.955 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T01:00:28.955 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-18T01:00:29.956 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-02-18T01:00:29.956 DEBUG:teuthology.orchestra.run.smithi105:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-18T01:00:29.998 INFO:teuthology.orchestra.run.smithi105.stdout:ttyS1 2024-02-18T01:00:30.015 DEBUG:teuthology.parallel:result is None 2024-02-18T01:00:39.360 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.155 2024-02-18T01:00:51.362 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T01:00:51.362 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T01:00:51.463 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-02-18T01:00:51.777 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T01:00:51.778 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.15.0-213-generic"... 2024-02-18T01:00:51.778 DEBUG:teuthology.orchestra.run.smithi155:> uname -r 2024-02-18T01:00:51.824 INFO:teuthology.orchestra.run.smithi155.stdout:4.15.0-213-generic 2024-02-18T01:00:51.825 DEBUG:teuthology.task.kernel:current kernel version is 4.15.0-213-generic vs 4.15.0-213-generic 2024-02-18T01:00:51.825 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-02-18T01:00:51.825 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-02-18T01:00:52.826 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-02-18T01:00:52.827 DEBUG:teuthology.orchestra.run.smithi155:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-02-18T01:00:52.855 INFO:teuthology.orchestra.run.smithi155.stdout:ttyS1 2024-02-18T01:00:52.880 DEBUG:teuthology.parallel:result is None 2024-02-18T01:00:52.881 INFO:teuthology.run_tasks:Running task internal.base... 2024-02-18T01:00:52.888 INFO:teuthology.task.internal:Creating test directory... 2024-02-18T01:00:52.889 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T01:00:52.891 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-02-18T01:00:52.928 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-02-18T01:00:52.934 INFO:teuthology.run_tasks:Running task internal.archive... 2024-02-18T01:00:52.944 INFO:teuthology.task.internal:Creating archive directory... 2024-02-18T01:00:52.944 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T01:00:52.947 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-02-18T01:00:52.989 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-02-18T01:00:52.996 INFO:teuthology.task.internal:Enabling coredump saving... 2024-02-18T01:00:52.996 DEBUG:teuthology.orchestra.run.smithi105:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T01:00:53.004 DEBUG:teuthology.orchestra.run.smithi155:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-02-18T01:00:53.022 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T01:00:53.030 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T01:00:53.050 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T01:00:53.059 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-02-18T01:00:53.061 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-02-18T01:00:53.068 INFO:teuthology.task.internal:Configuring sudo... 2024-02-18T01:00:53.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T01:00:53.076 DEBUG:teuthology.orchestra.run.smithi155:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-02-18T01:00:53.118 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-02-18T01:00:53.127 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-02-18T01:00:53.127 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T01:00:53.132 DEBUG:teuthology.orchestra.run.smithi155:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-02-18T01:00:53.168 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T01:00:53.188 DEBUG:teuthology.orchestra.run.smithi105:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T01:00:53.237 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:00:53.237 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T01:00:53.307 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-02-18T01:00:53.315 DEBUG:teuthology.orchestra.run.smithi155:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-02-18T01:00:53.364 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:00:53.364 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-02-18T01:00:53.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo service rsyslog restart 2024-02-18T01:00:53.443 DEBUG:teuthology.orchestra.run.smithi155:> sudo service rsyslog restart 2024-02-18T01:00:53.521 INFO:teuthology.run_tasks:Running task internal.timer... 2024-02-18T01:00:53.529 INFO:teuthology.task.internal:Starting timer... 2024-02-18T01:00:53.529 INFO:teuthology.run_tasks:Running task pcp... 2024-02-18T01:00:53.540 INFO:teuthology.run_tasks:Running task selinux... 2024-02-18T01:00:53.548 DEBUG:teuthology.task.selinux:Excluding smithi105: OS 'ubuntu' does not support SELinux 2024-02-18T01:00:53.549 DEBUG:teuthology.task.selinux:Excluding smithi155: OS 'ubuntu' does not support SELinux 2024-02-18T01:00:53.549 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-02-18T01:00:53.549 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-02-18T01:00:53.549 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-02-18T01:00:53.549 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-02-18T01:00:53.561 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-02-18T01:00:53.566 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-02-18T01:00:53.780 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-02-18T01:00:53.798 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-02-18T01:00:53.800 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi105.front.sepia.ceph.com,smithi155.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-02-18T01:06:19.071 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi105.front.sepia.ceph.com'), Remote(name='ubuntu@smithi155.front.sepia.ceph.com')] 2024-02-18T01:06:19.072 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-02-18T01:06:19.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi105.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T01:06:19.167 DEBUG:teuthology.orchestra.run.smithi105:> true 2024-02-18T01:06:19.285 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi105.front.sepia.ceph.com' 2024-02-18T01:06:19.285 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T01:06:19.286 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi155.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-02-18T01:06:19.370 DEBUG:teuthology.orchestra.run.smithi155:> true 2024-02-18T01:06:19.489 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi155.front.sepia.ceph.com' 2024-02-18T01:06:19.489 INFO:teuthology.run_tasks:Running task clock... 2024-02-18T01:06:19.538 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-02-18T01:06:19.538 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T01:06:19.540 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T01:06:19.542 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-02-18T01:06:19.542 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T01:06:19.594 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: ntpd 4.2.8p10@1.3728-o (1): Starting 2024-02-18T01:06:19.594 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Command line: ntpd -gq 2024-02-18T01:06:19.596 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: proto: precision = 0.123 usec (-23) 2024-02-18T01:06:19.596 INFO:teuthology.orchestra.run.smithi105.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-18T01:06:19.596 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-18T01:06:19.597 INFO:teuthology.orchestra.run.smithi105.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-18T01:06:19.598 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: restrict ::: KOD does nothing without LIMITED. 2024-02-18T01:06:19.598 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen and drop on 0 v6wildcard [::]:123 2024-02-18T01:06:19.598 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-18T01:06:19.598 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen normally on 2 lo 127.0.0.1:123 2024-02-18T01:06:19.599 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen normally on 3 enp3s0f1 172.21.15.105:123 2024-02-18T01:06:19.599 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen normally on 4 lo [::1]:123 2024-02-18T01:06:19.599 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d429%5]:123 2024-02-18T01:06:19.599 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:19 ntpd[14659]: Listening on routing socket on fd #22 for interface updates 2024-02-18T01:06:19.607 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: ntpd 4.2.8p10@1.3728-o (1): Starting 2024-02-18T01:06:19.608 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Command line: ntpd -gq 2024-02-18T01:06:19.608 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: proto: precision = 0.127 usec (-23) 2024-02-18T01:06:19.608 INFO:teuthology.orchestra.run.smithi155.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-18T01:06:19.609 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-02-18T01:06:19.609 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: restrict ::: KOD does nothing without LIMITED. 2024-02-18T01:06:19.610 INFO:teuthology.orchestra.run.smithi155.stderr:restrict ::: KOD does nothing without LIMITED. 2024-02-18T01:06:19.611 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen and drop on 0 v6wildcard [::]:123 2024-02-18T01:06:19.611 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-02-18T01:06:19.612 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen normally on 2 lo 127.0.0.1:123 2024-02-18T01:06:19.612 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen normally on 3 enp3s0f1 172.21.15.155:123 2024-02-18T01:06:19.612 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen normally on 4 lo [::1]:123 2024-02-18T01:06:19.613 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6ded%5]:123 2024-02-18T01:06:19.613 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:19 ntpd[14666]: Listening on routing socket on fd #22 for interface updates 2024-02-18T01:06:21.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:21 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:21.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:21 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:21.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:21 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:21.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:21 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:22.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:22 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:22.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:22 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:22.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:22 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:22.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:22 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:23 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:23 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:23 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:23.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:23 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:23.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:23 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:24.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:24.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:24 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:24.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:24 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:24.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:24 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:24.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:24 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:24.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:24 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:25.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:25 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:25.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:25 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:25.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:25 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:25.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:25 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:25.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:25.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:25.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:25.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:25.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:25.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:25.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:25.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:25 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:26.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:26.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:26 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:26.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:26 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:26.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:26 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:26.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:26 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:26.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:26 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:27.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:27 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:27.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:27 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:27.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:27 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:27.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:27 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:27.606 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stdout:18 Feb 01:06:27 ntpd[14666]: ntpd: time slew +0.002650 s 2024-02-18T01:06:27.607 INFO:teuthology.orchestra.run.smithi155.stdout:ntpd: time slew +0.002650s 2024-02-18T01:06:27.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-18T01:06:27.608 INFO:teuthology.orchestra.run.smithi155.stderr:18 Feb 01:06:27 ntpd[14666]: can't open /var/log/ntpstats/loopstats.20240218: Permission denied 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:27.657 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:28.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:28.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:28 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-02-18T01:06:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: can't open /var/log/ntpstats/rawstats.20240218: Permission denied 2024-02-18T01:06:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-02-18T01:06:29.595 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: can't open /var/log/ntpstats/peerstats.20240218: Permission denied 2024-02-18T01:06:29.596 INFO:teuthology.orchestra.run.smithi105.stdout:18 Feb 01:06:29 ntpd[14659]: ntpd: time slew -0.002711 s 2024-02-18T01:06:29.596 INFO:teuthology.orchestra.run.smithi105.stdout:ntpd: time slew -0.002711s 2024-02-18T01:06:29.596 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-02-18T01:06:29.597 INFO:teuthology.orchestra.run.smithi105.stderr:18 Feb 01:06:29 ntpd[14659]: can't open /var/log/ntpstats/loopstats.20240218: Permission denied 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:29.659 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-02-18T01:06:29.660 INFO:teuthology.run_tasks:Running task install... 2024-02-18T01:06:29.669 DEBUG:teuthology.task.install:project ceph 2024-02-18T01:06:29.669 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'}, 'extra_packages': ['cephadm']} 2024-02-18T01:06:29.669 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1'} 2024-02-18T01:06:29.669 INFO:teuthology.task.install:Using flavor: default 2024-02-18T01:06:29.675 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-02-18T01:06:29.675 INFO:teuthology.task.install:extra packages: [] 2024-02-18T01:06:29.676 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-key list | grep Ceph 2024-02-18T01:06:29.677 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-key list | grep Ceph 2024-02-18T01:06:29.772 INFO:teuthology.orchestra.run.smithi105.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-18T01:06:29.774 INFO:teuthology.orchestra.run.smithi155.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-02-18T01:06:29.828 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-18T01:06:29.828 INFO:teuthology.orchestra.run.smithi105.stdout:uid [ unknown] Ceph.com (release key) 2024-02-18T01:06:29.829 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-18T01:06:29.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:06:29.835 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-02-18T01:06:29.835 INFO:teuthology.orchestra.run.smithi155.stdout:uid [ unknown] Ceph.com (release key) 2024-02-18T01:06:29.835 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-02-18T01:06:29.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:06:29.980 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default/ 2024-02-18T01:06:29.980 INFO:teuthology.task.install.deb:Package version is 16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:06:30.060 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default/ 2024-02-18T01:06:30.060 INFO:teuthology.task.install.deb:Package version is 16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:06:30.092 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:06:30.093 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-18T01:06:30.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo apt-get update 2024-02-18T01:06:30.238 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:06:30.239 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-02-18T01:06:30.253 DEBUG:teuthology.orchestra.run.smithi155:> sudo apt-get update 2024-02-18T01:06:30.353 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic InRelease 2024-02-18T01:06:30.374 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2024-02-18T01:06:30.397 INFO:teuthology.orchestra.run.smithi105.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic Release.gpg 2024-02-18T01:06:30.423 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:4 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-18T01:06:30.424 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:5 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T01:06:30.424 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,125 B] 2024-02-18T01:06:30.446 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:1 http://security.ubuntu.com/ubuntu bionic-security InRelease 2024-02-18T01:06:30.456 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 Packages [17.2 kB] 2024-02-18T01:06:30.496 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-18T01:06:30.556 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:2 http://archive.ubuntu.com/ubuntu bionic InRelease 2024-02-18T01:06:30.561 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:3 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic InRelease 2024-02-18T01:06:30.578 INFO:teuthology.orchestra.run.smithi105.stdout:Hit:9 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-18T01:06:30.597 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic Release [7,681 B] 2024-02-18T01:06:30.636 INFO:teuthology.orchestra.run.smithi155.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic Release.gpg 2024-02-18T01:06:30.637 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:6 http://archive.ubuntu.com/ubuntu bionic-updates InRelease 2024-02-18T01:06:30.672 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main i386 Packages [3,125 B] 2024-02-18T01:06:30.719 INFO:teuthology.orchestra.run.smithi155.stdout:Hit:8 http://archive.ubuntu.com/ubuntu bionic-backports InRelease 2024-02-18T01:06:30.721 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 Packages [17.2 kB] 2024-02-18T01:06:30.998 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 28.0 kB in 1s (41.8 kB/s) 2024-02-18T01:06:31.038 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 28.0 kB in 1s (48.1 kB/s) 2024-02-18T01:06:32.484 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T01:06:32.509 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T01:06:32.537 DEBUG:teuthology.orchestra.run.smithi105:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-548-g81bd20d6-1bionic cephadm=16.2.14-548-g81bd20d6-1bionic ceph-mds=16.2.14-548-g81bd20d6-1bionic ceph-mgr=16.2.14-548-g81bd20d6-1bionic ceph-common=16.2.14-548-g81bd20d6-1bionic ceph-fuse=16.2.14-548-g81bd20d6-1bionic ceph-test=16.2.14-548-g81bd20d6-1bionic radosgw=16.2.14-548-g81bd20d6-1bionic python3-rados=16.2.14-548-g81bd20d6-1bionic python3-rgw=16.2.14-548-g81bd20d6-1bionic python3-cephfs=16.2.14-548-g81bd20d6-1bionic python3-rbd=16.2.14-548-g81bd20d6-1bionic libcephfs2=16.2.14-548-g81bd20d6-1bionic libcephfs-dev=16.2.14-548-g81bd20d6-1bionic librados2=16.2.14-548-g81bd20d6-1bionic librbd1=16.2.14-548-g81bd20d6-1bionic rbd-fuse=16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:06:32.559 DEBUG:teuthology.orchestra.run.smithi155:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=16.2.14-548-g81bd20d6-1bionic cephadm=16.2.14-548-g81bd20d6-1bionic ceph-mds=16.2.14-548-g81bd20d6-1bionic ceph-mgr=16.2.14-548-g81bd20d6-1bionic ceph-common=16.2.14-548-g81bd20d6-1bionic ceph-fuse=16.2.14-548-g81bd20d6-1bionic ceph-test=16.2.14-548-g81bd20d6-1bionic radosgw=16.2.14-548-g81bd20d6-1bionic python3-rados=16.2.14-548-g81bd20d6-1bionic python3-rgw=16.2.14-548-g81bd20d6-1bionic python3-cephfs=16.2.14-548-g81bd20d6-1bionic python3-rbd=16.2.14-548-g81bd20d6-1bionic libcephfs2=16.2.14-548-g81bd20d6-1bionic libcephfs-dev=16.2.14-548-g81bd20d6-1bionic librados2=16.2.14-548-g81bd20d6-1bionic librbd1=16.2.14-548-g81bd20d6-1bionic rbd-fuse=16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:06:32.632 INFO:teuthology.orchestra.run.smithi105.stdout:Reading package lists... 2024-02-18T01:06:32.640 INFO:teuthology.orchestra.run.smithi155.stdout:Reading package lists... 2024-02-18T01:06:32.902 INFO:teuthology.orchestra.run.smithi155.stdout:Building dependency tree... 2024-02-18T01:06:32.902 INFO:teuthology.orchestra.run.smithi155.stdout:Reading state information... 2024-02-18T01:06:32.917 INFO:teuthology.orchestra.run.smithi105.stdout:Building dependency tree... 2024-02-18T01:06:32.918 INFO:teuthology.orchestra.run.smithi105.stdout:Reading state information... 2024-02-18T01:06:33.107 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages were automatically installed and are no longer required: 2024-02-18T01:06:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-18T01:06:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-18T01:06:33.108 INFO:teuthology.orchestra.run.smithi155.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-18T01:06:33.108 INFO:teuthology.orchestra.run.smithi155.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-18T01:06:33.109 INFO:teuthology.orchestra.run.smithi155.stdout:The following additional packages will be installed: 2024-02-18T01:06:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-18T01:06:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-18T01:06:33.109 INFO:teuthology.orchestra.run.smithi155.stdout: libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 libonig4 2024-02-18T01:06:33.110 INFO:teuthology.orchestra.run.smithi155.stdout: librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2024-02-18T01:06:33.111 INFO:teuthology.orchestra.run.smithi155.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pastescript python3-pecan python3-pil python3-pkg-resources 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2024-02-18T01:06:33.112 INFO:teuthology.orchestra.run.smithi155.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout:Suggested packages: 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python-pytest-xdist python-mock python-cryptography-doc 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2024-02-18T01:06:33.113 INFO:teuthology.orchestra.run.smithi155.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2024-02-18T01:06:33.114 INFO:teuthology.orchestra.run.smithi155.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2024-02-18T01:06:33.124 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages were automatically installed and are no longer required: 2024-02-18T01:06:33.124 INFO:teuthology.orchestra.run.smithi105.stdout: linux-headers-4.15.0-194 linux-headers-4.15.0-194-generic 2024-02-18T01:06:33.124 INFO:teuthology.orchestra.run.smithi105.stdout: linux-image-4.15.0-194-generic linux-modules-4.15.0-194-generic 2024-02-18T01:06:33.125 INFO:teuthology.orchestra.run.smithi105.stdout: linux-modules-extra-4.15.0-194-generic 2024-02-18T01:06:33.125 INFO:teuthology.orchestra.run.smithi105.stdout:Use 'sudo apt autoremove' to remove them. 2024-02-18T01:06:33.125 INFO:teuthology.orchestra.run.smithi105.stdout:The following additional packages will be installed: 2024-02-18T01:06:33.125 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-02-18T01:06:33.126 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq libjaeger 2024-02-18T01:06:33.126 INFO:teuthology.orchestra.run.smithi105.stdout: libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 libonig4 2024-02-18T01:06:33.126 INFO:teuthology.orchestra.run.smithi105.stdout: librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-18T01:06:33.127 INFO:teuthology.orchestra.run.smithi105.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse python3-ceph-common python3-cffi-backend 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cherrypy3 python3-cryptography python3-cycler python3-dateutil 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-decorator python3-fasteners python3-html5lib python3-httplib2 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-matplotlib python3-monotonic python3-numpy python3-oauth2client 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-olefile python3-openssl python3-paste python3-pastedeploy 2024-02-18T01:06:33.128 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pastescript python3-pecan python3-pil python3-pkg-resources 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-repoze.lru python3-routes python3-rsa python3-scipy 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-setuptools python3-simplegeneric python3-simplejson 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-sqlalchemy python3-tempita python3-tk python3-tz python3-uritemplate 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-waitress python3-webencodings python3-webob python3-websocket 2024-02-18T01:06:33.129 INFO:teuthology.orchestra.run.smithi105.stdout: python3-webtest python3-werkzeug socat xmlstarlet 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout:Suggested packages: 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-influxdb python-attr-doc python-funcsigs-doc subversion 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python-pytest-xdist python-mock python-cryptography-doc 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cryptography-vectors python-cycler-doc python3-genshi 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python-jinja2-doc python3-crypto python3-lxml-dbg python-lxml-doc 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-beaker python-mako-doc dvipng ffmpeg gir1.2-gtk-3.0 ghostscript 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: inkscape ipython3 python-matplotlib-doc python3-cairocffi python3-gi-cairo 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-gobject python3-pyqt4 python3-sip python3-tornado 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: texlive-extra-utils texlive-latex-extra ttf-staypuft python-numpy-doc 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python3-dev python3-numpy-dbg python-openssl-doc python3-openssl-dbg 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python-pecan-doc python-pil-doc python3-pil-dbg python-pyinotify-doc 2024-02-18T01:06:33.130 INFO:teuthology.orchestra.run.smithi105.stdout: python-pyparsing-doc python-scipy-doc python-setuptools-doc python3-dap 2024-02-18T01:06:33.131 INFO:teuthology.orchestra.run.smithi105.stdout: python-sklearn-doc python-sqlalchemy-doc python3-psycopg2 python3-pymysql 2024-02-18T01:06:33.131 INFO:teuthology.orchestra.run.smithi105.stdout: python3-fdb tix python3-tk-dbg python-waitress-doc python-webob-doc 2024-02-18T01:06:33.131 INFO:teuthology.orchestra.run.smithi105.stdout: python-webtest-doc python3-termcolor python3-watchdog python-werkzeug-doc 2024-02-18T01:06:33.236 INFO:teuthology.orchestra.run.smithi155.stdout:The following NEW packages will be installed: 2024-02-18T01:06:33.237 INFO:teuthology.orchestra.run.smithi155.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-18T01:06:33.237 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-18T01:06:33.237 INFO:teuthology.orchestra.run.smithi155.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-18T01:06:33.239 INFO:teuthology.orchestra.run.smithi155.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-jwt 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2024-02-18T01:06:33.240 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2024-02-18T01:06:33.241 INFO:teuthology.orchestra.run.smithi155.stdout: xmlstarlet 2024-02-18T01:06:33.242 INFO:teuthology.orchestra.run.smithi155.stdout:The following packages will be upgraded: 2024-02-18T01:06:33.242 INFO:teuthology.orchestra.run.smithi155.stdout: librados2 librbd1 python3-pkg-resources 2024-02-18T01:06:33.246 INFO:teuthology.orchestra.run.smithi105.stdout:The following NEW packages will be installed: 2024-02-18T01:06:33.246 INFO:teuthology.orchestra.run.smithi105.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-02-18T01:06:33.246 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-02-18T01:06:33.247 INFO:teuthology.orchestra.run.smithi105.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test cephadm jq libcephfs-dev 2024-02-18T01:06:33.247 INFO:teuthology.orchestra.run.smithi105.stdout: libcephfs2 libjaeger libjq1 libleveldb1v5 liblttng-ust0 liblua5.3-0 liboath0 2024-02-18T01:06:33.247 INFO:teuthology.orchestra.run.smithi105.stdout: libonig4 librabbitmq4 libradosstriper1 librdkafka1 librgw2 libsnappy1v5 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: libsqlite3-mod-ceph libxslt1.1 nvme-cli python-attr python-funcsigs 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python-pastedeploy-tpl python-pluggy python-py python-pytest 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-asn1crypto python3-attr python3-bcrypt python3-bs4 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-cffi-backend python3-cherrypy3 python3-cryptography python3-cycler 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-dateutil python3-decorator python3-fasteners python3-html5lib 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-httplib2 python3-jinja2 python3-joblib python3-jwt 2024-02-18T01:06:33.249 INFO:teuthology.orchestra.run.smithi105.stdout: python3-kubernetes python3-logutils python3-lxml python3-mako 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-markupsafe python3-matplotlib python3-monotonic python3-numpy 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-oauth2client python3-olefile python3-openssl python3-paste 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pastedeploy python3-pastescript python3-pecan python3-pil 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pluggy python3-prettytable python3-py python3-pyasn1 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-pyasn1-modules python3-pyinotify python3-pyparsing python3-pytest 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rados python3-rbd python3-repoze.lru python3-rgw python3-routes 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-rsa python3-scipy python3-setuptools python3-simplegeneric 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-simplejson python3-singledispatch python3-sklearn 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-sklearn-lib python3-sqlalchemy python3-tempita python3-tk python3-tz 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-uritemplate python3-waitress python3-webencodings python3-webob 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: python3-websocket python3-webtest python3-werkzeug radosgw rbd-fuse socat 2024-02-18T01:06:33.250 INFO:teuthology.orchestra.run.smithi105.stdout: xmlstarlet 2024-02-18T01:06:33.251 INFO:teuthology.orchestra.run.smithi105.stdout:The following packages will be upgraded: 2024-02-18T01:06:33.251 INFO:teuthology.orchestra.run.smithi105.stdout: librados2 librbd1 python3-pkg-resources 2024-02-18T01:06:33.320 INFO:teuthology.orchestra.run.smithi155.stdout:3 upgraded, 110 newly installed, 0 to remove and 145 not upgraded. 2024-02-18T01:06:33.321 INFO:teuthology.orchestra.run.smithi155.stdout:Need to get 156 MB of archives. 2024-02-18T01:06:33.321 INFO:teuthology.orchestra.run.smithi155.stdout:After this operation, 690 MB of additional disk space will be used. 2024-02-18T01:06:33.321 INFO:teuthology.orchestra.run.smithi155.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2024-02-18T01:06:33.331 INFO:teuthology.orchestra.run.smithi105.stdout:3 upgraded, 110 newly installed, 0 to remove and 145 not upgraded. 2024-02-18T01:06:33.331 INFO:teuthology.orchestra.run.smithi105.stdout:Need to get 156 MB of archives. 2024-02-18T01:06:33.331 INFO:teuthology.orchestra.run.smithi105.stdout:After this operation, 690 MB of additional disk space will be used. 2024-02-18T01:06:33.331 INFO:teuthology.orchestra.run.smithi105.stdout:Get:1 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-uritemplate all 0.6-1ubuntu1 [5,126 B] 2024-02-18T01:06:33.341 INFO:teuthology.orchestra.run.smithi155.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2024-02-18T01:06:33.352 INFO:teuthology.orchestra.run.smithi105.stdout:Get:2 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liblttng-ust0 amd64 2.10.1-1 [154 kB] 2024-02-18T01:06:33.369 INFO:teuthology.orchestra.run.smithi155.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.2.14-548-g81bd20d6-1bionic [3,964 B] 2024-02-18T01:06:33.381 INFO:teuthology.orchestra.run.smithi105.stdout:Get:3 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libjaeger amd64 16.2.14-548-g81bd20d6-1bionic [3,964 B] 2024-02-18T01:06:33.411 INFO:teuthology.orchestra.run.smithi155.stdout:Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2024-02-18T01:06:33.411 INFO:teuthology.orchestra.run.smithi155.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.2.14-548-g81bd20d6-1bionic [3,175 kB] 2024-02-18T01:06:33.423 INFO:teuthology.orchestra.run.smithi105.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librbd1 amd64 16.2.14-548-g81bd20d6-1bionic [3,175 kB] 2024-02-18T01:06:33.423 INFO:teuthology.orchestra.run.smithi155.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblua5.3-0 amd64 5.3.3-1ubuntu0.18.04.1 [115 kB] 2024-02-18T01:06:33.428 INFO:teuthology.orchestra.run.smithi105.stdout:Get:5 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-prettytable all 0.7.2-3 [19.7 kB] 2024-02-18T01:06:33.431 INFO:teuthology.orchestra.run.smithi155.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2024-02-18T01:06:33.432 INFO:teuthology.orchestra.run.smithi105.stdout:Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 liblua5.3-0 amd64 5.3.3-1ubuntu0.18.04.1 [115 kB] 2024-02-18T01:06:33.435 INFO:teuthology.orchestra.run.smithi155.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2024-02-18T01:06:33.453 INFO:teuthology.orchestra.run.smithi105.stdout:Get:7 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 librabbitmq4 amd64 0.8.0-1ubuntu0.18.04.2 [33.9 kB] 2024-02-18T01:06:33.456 INFO:teuthology.orchestra.run.smithi105.stdout:Get:8 http://archive.ubuntu.com/ubuntu bionic/universe amd64 librdkafka1 amd64 0.11.3-1build1 [293 kB] 2024-02-18T01:06:33.459 INFO:teuthology.orchestra.run.smithi155.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2024-02-18T01:06:33.459 INFO:teuthology.orchestra.run.smithi155.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2024-02-18T01:06:33.477 INFO:teuthology.orchestra.run.smithi155.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2024-02-18T01:06:33.482 INFO:teuthology.orchestra.run.smithi155.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2024-02-18T01:06:33.484 INFO:teuthology.orchestra.run.smithi155.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2024-02-18T01:06:33.486 INFO:teuthology.orchestra.run.smithi155.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2024-02-18T01:06:33.487 INFO:teuthology.orchestra.run.smithi105.stdout:Get:9 http://archive.ubuntu.com/ubuntu bionic/main amd64 libsnappy1v5 amd64 1.1.7-1 [16.0 kB] 2024-02-18T01:06:33.488 INFO:teuthology.orchestra.run.smithi155.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2024-02-18T01:06:33.488 INFO:teuthology.orchestra.run.smithi105.stdout:Get:10 http://archive.ubuntu.com/ubuntu bionic/main amd64 libleveldb1v5 amd64 1.20-2 [136 kB] 2024-02-18T01:06:33.492 INFO:teuthology.orchestra.run.smithi105.stdout:Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 liboath0 amd64 2.6.1-1 [44.7 kB] 2024-02-18T01:06:33.494 INFO:teuthology.orchestra.run.smithi155.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2024-02-18T01:06:33.495 INFO:teuthology.orchestra.run.smithi155.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2024-02-18T01:06:33.496 INFO:teuthology.orchestra.run.smithi105.stdout:Get:12 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-dateutil all 2.6.1-1 [52.3 kB] 2024-02-18T01:06:33.500 INFO:teuthology.orchestra.run.smithi105.stdout:Get:13 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-asn1crypto all 0.24.0-1 [72.8 kB] 2024-02-18T01:06:33.503 INFO:teuthology.orchestra.run.smithi105.stdout:Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-cffi-backend amd64 1.11.5-1 [64.6 kB] 2024-02-18T01:06:33.506 INFO:teuthology.orchestra.run.smithi105.stdout:Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-cryptography amd64 2.1.4-1ubuntu1.4 [220 kB] 2024-02-18T01:06:33.506 INFO:teuthology.orchestra.run.smithi155.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2024-02-18T01:06:33.514 INFO:teuthology.orchestra.run.smithi155.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jwt all 1.5.3+ds1-1ubuntu0.1 [16.6 kB] 2024-02-18T01:06:33.517 INFO:teuthology.orchestra.run.smithi155.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2024-02-18T01:06:33.517 INFO:teuthology.orchestra.run.smithi155.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2024-02-18T01:06:33.518 INFO:teuthology.orchestra.run.smithi155.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-mako all 1.0.7+ds1-1ubuntu0.2 [59.4 kB] 2024-02-18T01:06:33.525 INFO:teuthology.orchestra.run.smithi105.stdout:Get:16 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-openssl all 17.5.0-1ubuntu1 [41.5 kB] 2024-02-18T01:06:33.525 INFO:teuthology.orchestra.run.smithi105.stdout:Get:17 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-bcrypt amd64 3.1.4-2 [29.9 kB] 2024-02-18T01:06:33.526 INFO:teuthology.orchestra.run.smithi105.stdout:Get:18 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cherrypy3 all 8.9.1-2 [160 kB] 2024-02-18T01:06:33.526 INFO:teuthology.orchestra.run.smithi155.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2024-02-18T01:06:33.527 INFO:teuthology.orchestra.run.smithi155.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2024-02-18T01:06:33.527 INFO:teuthology.orchestra.run.smithi155.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2024-02-18T01:06:33.529 INFO:teuthology.orchestra.run.smithi155.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2024-02-18T01:06:33.530 INFO:teuthology.orchestra.run.smithi155.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2024-02-18T01:06:33.531 INFO:teuthology.orchestra.run.smithi105.stdout:Get:19 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jwt all 1.5.3+ds1-1ubuntu0.1 [16.6 kB] 2024-02-18T01:06:33.532 INFO:teuthology.orchestra.run.smithi105.stdout:Get:20 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-logutils all 0.3.3-5 [16.7 kB] 2024-02-18T01:06:33.533 INFO:teuthology.orchestra.run.smithi105.stdout:Get:21 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-markupsafe amd64 1.0-1build1 [13.5 kB] 2024-02-18T01:06:33.533 INFO:teuthology.orchestra.run.smithi105.stdout:Get:22 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-mako all 1.0.7+ds1-1ubuntu0.2 [59.4 kB] 2024-02-18T01:06:33.537 INFO:teuthology.orchestra.run.smithi105.stdout:Get:23 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-simplegeneric all 0.8.1-1 [11.5 kB] 2024-02-18T01:06:33.546 INFO:teuthology.orchestra.run.smithi155.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pkg-resources all 39.0.1-2ubuntu0.1 [99.0 kB] 2024-02-18T01:06:33.546 INFO:teuthology.orchestra.run.smithi105.stdout:Get:24 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-singledispatch all 3.4.0.3-2 [7,022 B] 2024-02-18T01:06:33.547 INFO:teuthology.orchestra.run.smithi105.stdout:Get:25 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webob all 1:1.7.3-2fakesync1 [64.3 kB] 2024-02-18T01:06:33.548 INFO:teuthology.orchestra.run.smithi105.stdout:Get:26 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-bs4 all 4.6.0-1 [67.8 kB] 2024-02-18T01:06:33.549 INFO:teuthology.orchestra.run.smithi155.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2024-02-18T01:06:33.549 INFO:teuthology.orchestra.run.smithi155.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2024-02-18T01:06:33.550 INFO:teuthology.orchestra.run.smithi105.stdout:Get:27 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-waitress all 1.0.1-1 [53.4 kB] 2024-02-18T01:06:33.557 INFO:teuthology.orchestra.run.smithi155.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2024-02-18T01:06:33.557 INFO:teuthology.orchestra.run.smithi155.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2024-02-18T01:06:33.574 INFO:teuthology.orchestra.run.smithi105.stdout:Get:28 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pkg-resources all 39.0.1-2ubuntu0.1 [99.0 kB] 2024-02-18T01:06:33.575 INFO:teuthology.orchestra.run.smithi155.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2024-02-18T01:06:33.576 INFO:teuthology.orchestra.run.smithi155.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2024-02-18T01:06:33.577 INFO:teuthology.orchestra.run.smithi105.stdout:Get:29 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tempita all 0.5.2-2 [13.9 kB] 2024-02-18T01:06:33.577 INFO:teuthology.orchestra.run.smithi105.stdout:Get:30 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-paste all 2.0.3+dfsg-4ubuntu1 [456 kB] 2024-02-18T01:06:33.578 INFO:teuthology.orchestra.run.smithi155.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.2 [175 kB] 2024-02-18T01:06:33.582 INFO:teuthology.orchestra.run.smithi155.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2024-02-18T01:06:33.584 INFO:teuthology.orchestra.run.smithi155.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2024-02-18T01:06:33.588 INFO:teuthology.orchestra.run.smithi105.stdout:Get:31 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-pastedeploy-tpl all 1.5.2-4 [4,796 B] 2024-02-18T01:06:33.588 INFO:teuthology.orchestra.run.smithi105.stdout:Get:32 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastedeploy all 1.5.2-4 [13.4 kB] 2024-02-18T01:06:33.588 INFO:teuthology.orchestra.run.smithi105.stdout:Get:33 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webtest all 2.0.28-1ubuntu1 [27.9 kB] 2024-02-18T01:06:33.595 INFO:teuthology.orchestra.run.smithi105.stdout:Get:34 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pecan all 1.2.1-2 [86.1 kB] 2024-02-18T01:06:33.595 INFO:teuthology.orchestra.run.smithi155.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2024-02-18T01:06:33.596 INFO:teuthology.orchestra.run.smithi105.stdout:Get:35 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-werkzeug all 0.14.1+dfsg1-1ubuntu0.2 [175 kB] 2024-02-18T01:06:33.596 INFO:teuthology.orchestra.run.smithi155.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2024-02-18T01:06:33.600 INFO:teuthology.orchestra.run.smithi105.stdout:Get:36 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-jinja2 all 2.10-1ubuntu0.18.04.1 [95.4 kB] 2024-02-18T01:06:33.603 INFO:teuthology.orchestra.run.smithi105.stdout:Get:37 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-repoze.lru all 0.7-1 [11.7 kB] 2024-02-18T01:06:33.631 INFO:teuthology.orchestra.run.smithi105.stdout:Get:38 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-routes all 2.4.1-1 [28.4 kB] 2024-02-18T01:06:33.631 INFO:teuthology.orchestra.run.smithi105.stdout:Get:39 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-numpy amd64 1:1.13.3-2ubuntu1 [1,943 kB] 2024-02-18T01:06:33.638 INFO:teuthology.orchestra.run.smithi155.stdout:Get:40 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.2.14-548-g81bd20d6-1bionic [3,206 kB] 2024-02-18T01:06:33.645 INFO:teuthology.orchestra.run.smithi155.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2024-02-18T01:06:33.645 INFO:teuthology.orchestra.run.smithi155.stdout:Get:42 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2024-02-18T01:06:33.678 INFO:teuthology.orchestra.run.smithi105.stdout:Get:40 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-decorator all 4.1.2-1 [9,364 B] 2024-02-18T01:06:33.678 INFO:teuthology.orchestra.run.smithi105.stdout:Get:41 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-scipy amd64 0.19.1-2ubuntu1 [9,619 kB] 2024-02-18T01:06:33.680 INFO:teuthology.orchestra.run.smithi105.stdout:Get:42 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librados2 amd64 16.2.14-548-g81bd20d6-1bionic [3,206 kB] 2024-02-18T01:06:33.841 INFO:teuthology.orchestra.run.smithi155.stdout:Get:43 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.2.14-548-g81bd20d6-1bionic [684 kB] 2024-02-18T01:06:33.895 INFO:teuthology.orchestra.run.smithi155.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.2.14-548-g81bd20d6-1bionic [342 kB] 2024-02-18T01:06:33.907 INFO:teuthology.orchestra.run.smithi155.stdout:Get:45 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.2.14-548-g81bd20d6-1bionic [22.2 kB] 2024-02-18T01:06:33.908 INFO:teuthology.orchestra.run.smithi155.stdout:Get:46 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.2.14-548-g81bd20d6-1bionic [178 kB] 2024-02-18T01:06:33.913 INFO:teuthology.orchestra.run.smithi155.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.2.14-548-g81bd20d6-1bionic [38.4 kB] 2024-02-18T01:06:33.914 INFO:teuthology.orchestra.run.smithi155.stdout:Get:48 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.2.14-548-g81bd20d6-1bionic [336 kB] 2024-02-18T01:06:33.923 INFO:teuthology.orchestra.run.smithi105.stdout:Get:43 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2024-02-18T01:06:33.931 INFO:teuthology.orchestra.run.smithi105.stdout:Get:44 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs2 amd64 16.2.14-548-g81bd20d6-1bionic [684 kB] 2024-02-18T01:06:33.940 INFO:teuthology.orchestra.run.smithi105.stdout:Get:45 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2024-02-18T01:06:33.941 INFO:teuthology.orchestra.run.smithi105.stdout:Get:46 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2024-02-18T01:06:33.953 INFO:teuthology.orchestra.run.smithi105.stdout:Get:47 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rados amd64 16.2.14-548-g81bd20d6-1bionic [342 kB] 2024-02-18T01:06:33.958 INFO:teuthology.orchestra.run.smithi155.stdout:Get:49 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.2.14-548-g81bd20d6-1bionic [3,435 kB] 2024-02-18T01:06:33.961 INFO:teuthology.orchestra.run.smithi105.stdout:Get:48 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2024-02-18T01:06:33.961 INFO:teuthology.orchestra.run.smithi105.stdout:Get:49 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2024-02-18T01:06:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2024-02-18T01:06:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2024-02-18T01:06:33.962 INFO:teuthology.orchestra.run.smithi105.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2024-02-18T01:06:33.963 INFO:teuthology.orchestra.run.smithi105.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2024-02-18T01:06:33.967 INFO:teuthology.orchestra.run.smithi105.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2024-02-18T01:06:33.968 INFO:teuthology.orchestra.run.smithi155.stdout:Get:50 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn-lib amd64 0.19.1-3 [1,429 kB] 2024-02-18T01:06:33.983 INFO:teuthology.orchestra.run.smithi155.stdout:Get:51 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-joblib all 0.11-1 [109 kB] 2024-02-18T01:06:33.985 INFO:teuthology.orchestra.run.smithi155.stdout:Get:52 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-sklearn all 0.19.1-3 [1,373 kB] 2024-02-18T01:06:34.001 INFO:teuthology.orchestra.run.smithi105.stdout:Get:55 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-argparse all 16.2.14-548-g81bd20d6-1bionic [22.2 kB] 2024-02-18T01:06:34.002 INFO:teuthology.orchestra.run.smithi105.stdout:Get:56 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-cephfs amd64 16.2.14-548-g81bd20d6-1bionic [178 kB] 2024-02-18T01:06:34.008 INFO:teuthology.orchestra.run.smithi105.stdout:Get:57 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-ceph-common all 16.2.14-548-g81bd20d6-1bionic [38.4 kB] 2024-02-18T01:06:34.008 INFO:teuthology.orchestra.run.smithi105.stdout:Get:58 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rbd amd64 16.2.14-548-g81bd20d6-1bionic [336 kB] 2024-02-18T01:06:34.011 INFO:teuthology.orchestra.run.smithi105.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2024-02-18T01:06:34.013 INFO:teuthology.orchestra.run.smithi105.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-setuptools all 39.0.1-2ubuntu0.1 [248 kB] 2024-02-18T01:06:34.015 INFO:teuthology.orchestra.run.smithi155.stdout:Get:53 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-monotonic all 1.1-2 [5,332 B] 2024-02-18T01:06:34.015 INFO:teuthology.orchestra.run.smithi155.stdout:Get:54 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-fasteners all 0.12.0-3 [13.5 kB] 2024-02-18T01:06:34.016 INFO:teuthology.orchestra.run.smithi155.stdout:Get:55 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-httplib2 all 0.9.2+dfsg-1ubuntu0.3 [28.8 kB] 2024-02-18T01:06:34.017 INFO:teuthology.orchestra.run.smithi155.stdout:Get:56 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1 all 0.4.2-3 [46.8 kB] 2024-02-18T01:06:34.018 INFO:teuthology.orchestra.run.smithi105.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2024-02-18T01:06:34.018 INFO:teuthology.orchestra.run.smithi155.stdout:Get:57 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyasn1-modules all 0.2.1-0.2 [32.9 kB] 2024-02-18T01:06:34.018 INFO:teuthology.orchestra.run.smithi105.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2024-02-18T01:06:34.019 INFO:teuthology.orchestra.run.smithi155.stdout:Get:58 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-rsa all 3.4.2-1 [29.0 kB] 2024-02-18T01:06:34.020 INFO:teuthology.orchestra.run.smithi155.stdout:Get:59 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-sqlalchemy all 1.1.11+ds1-1ubuntu1 [680 kB] 2024-02-18T01:06:34.020 INFO:teuthology.orchestra.run.smithi105.stdout:Get:63 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 librgw2 amd64 16.2.14-548-g81bd20d6-1bionic [3,435 kB] 2024-02-18T01:06:34.022 INFO:teuthology.orchestra.run.smithi105.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2024-02-18T01:06:34.025 INFO:teuthology.orchestra.run.smithi105.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2024-02-18T01:06:34.026 INFO:teuthology.orchestra.run.smithi105.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2024-02-18T01:06:34.027 INFO:teuthology.orchestra.run.smithi105.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2024-02-18T01:06:34.032 INFO:teuthology.orchestra.run.smithi105.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.3 [150 kB] 2024-02-18T01:06:34.035 INFO:teuthology.orchestra.run.smithi105.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2024-02-18T01:06:34.041 INFO:teuthology.orchestra.run.smithi155.stdout:Get:60 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-oauth2client all 4.1.2-3 [61.5 kB] 2024-02-18T01:06:34.043 INFO:teuthology.orchestra.run.smithi155.stdout:Get:61 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-setuptools all 39.0.1-2ubuntu0.1 [248 kB] 2024-02-18T01:06:34.052 INFO:teuthology.orchestra.run.smithi155.stdout:Get:62 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-websocket all 0.44.0-0ubuntu2 [30.7 kB] 2024-02-18T01:06:34.053 INFO:teuthology.orchestra.run.smithi155.stdout:Get:63 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-kubernetes all 2.0.0-2ubuntu1 [181 kB] 2024-02-18T01:06:34.060 INFO:teuthology.orchestra.run.smithi105.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.2 [184 kB] 2024-02-18T01:06:34.069 INFO:teuthology.orchestra.run.smithi105.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2024-02-18T01:06:34.069 INFO:teuthology.orchestra.run.smithi105.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2024-02-18T01:06:34.070 INFO:teuthology.orchestra.run.smithi155.stdout:Get:64 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libonig4 amd64 6.7.0-1 [119 kB] 2024-02-18T01:06:34.071 INFO:teuthology.orchestra.run.smithi105.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2024-02-18T01:06:34.071 INFO:teuthology.orchestra.run.smithi105.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-18T01:06:34.072 INFO:teuthology.orchestra.run.smithi105.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2024-02-18T01:06:34.073 INFO:teuthology.orchestra.run.smithi155.stdout:Get:65 http://archive.ubuntu.com/ubuntu bionic/universe amd64 libjq1 amd64 1.5+dfsg-2 [111 kB] 2024-02-18T01:06:34.074 INFO:teuthology.orchestra.run.smithi105.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2024-02-18T01:06:34.074 INFO:teuthology.orchestra.run.smithi105.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2024-02-18T01:06:34.076 INFO:teuthology.orchestra.run.smithi155.stdout:Get:66 http://archive.ubuntu.com/ubuntu bionic/universe amd64 jq amd64 1.5+dfsg-2 [45.6 kB] 2024-02-18T01:06:34.078 INFO:teuthology.orchestra.run.smithi155.stdout:Get:67 http://archive.ubuntu.com/ubuntu bionic/main amd64 socat amd64 1.7.3.2-2ubuntu2 [342 kB] 2024-02-18T01:06:34.083 INFO:teuthology.orchestra.run.smithi155.stdout:Get:68 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libxslt1.1 amd64 1.1.29-5ubuntu0.3 [150 kB] 2024-02-18T01:06:34.084 INFO:teuthology.orchestra.run.smithi105.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2024-02-18T01:06:34.084 INFO:teuthology.orchestra.run.smithi105.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2024-02-18T01:06:34.085 INFO:teuthology.orchestra.run.smithi155.stdout:Get:69 http://archive.ubuntu.com/ubuntu bionic/universe amd64 xmlstarlet amd64 1.6.1-2 [382 kB] 2024-02-18T01:06:34.102 INFO:teuthology.orchestra.run.smithi155.stdout:Get:70 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 nvme-cli amd64 1.5-1ubuntu1.2 [184 kB] 2024-02-18T01:06:34.107 INFO:teuthology.orchestra.run.smithi105.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.6 [1,098 kB] 2024-02-18T01:06:34.110 INFO:teuthology.orchestra.run.smithi155.stdout:Get:71 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-attr all 17.4.0-2 [23.7 kB] 2024-02-18T01:06:34.110 INFO:teuthology.orchestra.run.smithi155.stdout:Get:72 http://archive.ubuntu.com/ubuntu bionic/main amd64 python-funcsigs all 1.0.2-4 [13.5 kB] 2024-02-18T01:06:34.110 INFO:teuthology.orchestra.run.smithi155.stdout:Get:73 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pluggy all 0.6.0-1 [12.2 kB] 2024-02-18T01:06:34.121 INFO:teuthology.orchestra.run.smithi155.stdout:Get:74 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-18T01:06:34.123 INFO:teuthology.orchestra.run.smithi155.stdout:Get:75 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python-pytest all 3.3.2-2 [142 kB] 2024-02-18T01:06:34.127 INFO:teuthology.orchestra.run.smithi105.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2024-02-18T01:06:34.127 INFO:teuthology.orchestra.run.smithi105.stdout:Get:82 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2024-02-18T01:06:34.127 INFO:teuthology.orchestra.run.smithi105.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2024-02-18T01:06:34.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:76 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-attr all 17.4.0-2 [23.8 kB] 2024-02-18T01:06:34.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:77 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-cycler all 0.10.0-1 [7,622 B] 2024-02-18T01:06:34.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:78 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-webencodings all 0.5-2 [10.4 kB] 2024-02-18T01:06:34.128 INFO:teuthology.orchestra.run.smithi155.stdout:Get:79 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-html5lib all 0.999999999-1 [81.9 kB] 2024-02-18T01:06:34.141 INFO:teuthology.orchestra.run.smithi155.stdout:Get:80 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-lxml amd64 4.2.1-1ubuntu0.6 [1,098 kB] 2024-02-18T01:06:34.167 INFO:teuthology.orchestra.run.smithi155.stdout:Get:81 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyparsing all 2.2.0+dfsg1-2 [52.2 kB] 2024-02-18T01:06:34.170 INFO:teuthology.orchestra.run.smithi155.stdout:Get:82 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.2.14-548-g81bd20d6-1bionic [99.5 kB] 2024-02-18T01:06:34.171 INFO:teuthology.orchestra.run.smithi155.stdout:Get:83 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-tz all 2018.3-2 [25.1 kB] 2024-02-18T01:06:34.171 INFO:teuthology.orchestra.run.smithi155.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-matplotlib amd64 2.1.1-2ubuntu3 [3,907 kB] 2024-02-18T01:06:34.173 INFO:teuthology.orchestra.run.smithi155.stdout:Get:85 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.2.14-548-g81bd20d6-1bionic [416 kB] 2024-02-18T01:06:34.185 INFO:teuthology.orchestra.run.smithi155.stdout:Get:86 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.2.14-548-g81bd20d6-1bionic [20.4 MB] 2024-02-18T01:06:34.207 INFO:teuthology.orchestra.run.smithi105.stdout:Get:84 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2024-02-18T01:06:34.208 INFO:teuthology.orchestra.run.smithi105.stdout:Get:85 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2024-02-18T01:06:34.210 INFO:teuthology.orchestra.run.smithi105.stdout:Get:86 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.8 [331 kB] 2024-02-18T01:06:34.217 INFO:teuthology.orchestra.run.smithi105.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2024-02-18T01:06:34.217 INFO:teuthology.orchestra.run.smithi105.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-18T01:06:34.218 INFO:teuthology.orchestra.run.smithi105.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2024-02-18T01:06:34.218 INFO:teuthology.orchestra.run.smithi105.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2024-02-18T01:06:34.220 INFO:teuthology.orchestra.run.smithi105.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2024-02-18T01:06:34.220 INFO:teuthology.orchestra.run.smithi105.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2024-02-18T01:06:34.260 INFO:teuthology.orchestra.run.smithi155.stdout:Get:87 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-olefile all 0.45.1-1 [33.3 kB] 2024-02-18T01:06:34.260 INFO:teuthology.orchestra.run.smithi155.stdout:Get:88 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pastescript all 2.0.2-2 [54.6 kB] 2024-02-18T01:06:34.261 INFO:teuthology.orchestra.run.smithi155.stdout:Get:89 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-pil amd64 5.1.0-1ubuntu0.8 [331 kB] 2024-02-18T01:06:34.266 INFO:teuthology.orchestra.run.smithi155.stdout:Get:90 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pluggy all 0.6.0-1 [12.3 kB] 2024-02-18T01:06:34.266 INFO:teuthology.orchestra.run.smithi155.stdout:Get:91 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 python3-py all 1.5.2-1ubuntu0.1 [65.3 kB] 2024-02-18T01:06:34.267 INFO:teuthology.orchestra.run.smithi155.stdout:Get:92 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-pyinotify all 0.9.6-1 [24.7 kB] 2024-02-18T01:06:34.267 INFO:teuthology.orchestra.run.smithi155.stdout:Get:93 http://archive.ubuntu.com/ubuntu bionic/universe amd64 python3-pytest all 3.3.2-2 [142 kB] 2024-02-18T01:06:34.268 INFO:teuthology.orchestra.run.smithi155.stdout:Get:94 http://archive.ubuntu.com/ubuntu bionic/main amd64 python3-simplejson amd64 3.13.2-1 [49.6 kB] 2024-02-18T01:06:34.269 INFO:teuthology.orchestra.run.smithi155.stdout:Get:95 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 python3-tk amd64 3.6.9-1~18.04 [119 kB] 2024-02-18T01:06:34.280 INFO:teuthology.orchestra.run.smithi105.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 python3-rgw amd64 16.2.14-548-g81bd20d6-1bionic [99.5 kB] 2024-02-18T01:06:34.282 INFO:teuthology.orchestra.run.smithi105.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libradosstriper1 amd64 16.2.14-548-g81bd20d6-1bionic [416 kB] 2024-02-18T01:06:34.296 INFO:teuthology.orchestra.run.smithi105.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-common amd64 16.2.14-548-g81bd20d6-1bionic [20.4 MB] 2024-02-18T01:06:35.447 INFO:teuthology.orchestra.run.smithi155.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.2.14-548-g81bd20d6-1bionic [4,564 kB] 2024-02-18T01:06:35.727 INFO:teuthology.orchestra.run.smithi155.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.2.14-548-g81bd20d6-1bionic [204 kB] 2024-02-18T01:06:35.733 INFO:teuthology.orchestra.run.smithi155.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libsqlite3-mod-ceph amd64 16.2.14-548-g81bd20d6-1bionic [122 kB] 2024-02-18T01:06:35.737 INFO:teuthology.orchestra.run.smithi155.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.2.14-548-g81bd20d6-1bionic [1,441 kB] 2024-02-18T01:06:35.746 INFO:teuthology.orchestra.run.smithi105.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-base amd64 16.2.14-548-g81bd20d6-1bionic [4,564 kB] 2024-02-18T01:06:35.851 INFO:teuthology.orchestra.run.smithi155.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.2.14-548-g81bd20d6-1bionic [3,786 kB] 2024-02-18T01:06:36.040 INFO:teuthology.orchestra.run.smithi105.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-modules-core all 16.2.14-548-g81bd20d6-1bionic [204 kB] 2024-02-18T01:06:36.046 INFO:teuthology.orchestra.run.smithi105.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libsqlite3-mod-ceph amd64 16.2.14-548-g81bd20d6-1bionic [122 kB] 2024-02-18T01:06:36.050 INFO:teuthology.orchestra.run.smithi105.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr amd64 16.2.14-548-g81bd20d6-1bionic [1,441 kB] 2024-02-18T01:06:36.075 INFO:teuthology.orchestra.run.smithi155.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.2.14-548-g81bd20d6-1bionic [20.1 MB] 2024-02-18T01:06:36.171 INFO:teuthology.orchestra.run.smithi105.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mon amd64 16.2.14-548-g81bd20d6-1bionic [3,786 kB] 2024-02-18T01:06:36.445 INFO:teuthology.orchestra.run.smithi105.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-osd amd64 16.2.14-548-g81bd20d6-1bionic [20.1 MB] 2024-02-18T01:06:37.339 INFO:teuthology.orchestra.run.smithi155.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.2.14-548-g81bd20d6-1bionic [4,060 B] 2024-02-18T01:06:37.339 INFO:teuthology.orchestra.run.smithi155.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.2.14-548-g81bd20d6-1bionic [791 kB] 2024-02-18T01:06:37.364 INFO:teuthology.orchestra.run.smithi155.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.2.14-548-g81bd20d6-1bionic [2,201 kB] 2024-02-18T01:06:37.503 INFO:teuthology.orchestra.run.smithi155.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.2.14-548-g81bd20d6-1bionic [77.4 kB] 2024-02-18T01:06:37.505 INFO:teuthology.orchestra.run.smithi155.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.2.14-548-g81bd20d6-1bionic [100 kB] 2024-02-18T01:06:37.508 INFO:teuthology.orchestra.run.smithi155.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.2.14-548-g81bd20d6-1bionic [2,824 kB] 2024-02-18T01:06:37.703 INFO:teuthology.orchestra.run.smithi155.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.2.14-548-g81bd20d6-1bionic [7,889 kB] 2024-02-18T01:06:37.822 INFO:teuthology.orchestra.run.smithi105.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph amd64 16.2.14-548-g81bd20d6-1bionic [4,060 B] 2024-02-18T01:06:37.822 INFO:teuthology.orchestra.run.smithi105.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-fuse amd64 16.2.14-548-g81bd20d6-1bionic [791 kB] 2024-02-18T01:06:37.884 INFO:teuthology.orchestra.run.smithi105.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mds amd64 16.2.14-548-g81bd20d6-1bionic [2,201 kB] 2024-02-18T01:06:38.033 INFO:teuthology.orchestra.run.smithi105.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 cephadm amd64 16.2.14-548-g81bd20d6-1bionic [77.4 kB] 2024-02-18T01:06:38.036 INFO:teuthology.orchestra.run.smithi105.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-cephadm all 16.2.14-548-g81bd20d6-1bionic [100 kB] 2024-02-18T01:06:38.039 INFO:teuthology.orchestra.run.smithi105.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-dashboard all 16.2.14-548-g81bd20d6-1bionic [2,824 kB] 2024-02-18T01:06:38.200 INFO:teuthology.orchestra.run.smithi155.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.2.14-548-g81bd20d6-1bionic [4,176 B] 2024-02-18T01:06:38.201 INFO:teuthology.orchestra.run.smithi155.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.2.14-548-g81bd20d6-1bionic [42.8 MB] 2024-02-18T01:06:38.246 INFO:teuthology.orchestra.run.smithi105.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-diskprediction-local all 16.2.14-548-g81bd20d6-1bionic [7,889 kB] 2024-02-18T01:06:38.776 INFO:teuthology.orchestra.run.smithi105.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-mgr-k8sevents all 16.2.14-548-g81bd20d6-1bionic [4,176 B] 2024-02-18T01:06:38.776 INFO:teuthology.orchestra.run.smithi105.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 ceph-test amd64 16.2.14-548-g81bd20d6-1bionic [42.8 MB] 2024-02-18T01:06:40.826 INFO:teuthology.orchestra.run.smithi155.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.2.14-548-g81bd20d6-1bionic [21.5 kB] 2024-02-18T01:06:40.827 INFO:teuthology.orchestra.run.smithi155.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.2.14-548-g81bd20d6-1bionic [10.5 MB] 2024-02-18T01:06:41.363 INFO:teuthology.orchestra.run.smithi155.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.2.14-548-g81bd20d6-1bionic [78.6 kB] 2024-02-18T01:06:41.633 INFO:teuthology.orchestra.run.smithi105.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 libcephfs-dev amd64 16.2.14-548-g81bd20d6-1bionic [21.5 kB] 2024-02-18T01:06:41.634 INFO:teuthology.orchestra.run.smithi105.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 radosgw amd64 16.2.14-548-g81bd20d6-1bionic [10.5 MB] 2024-02-18T01:06:41.973 INFO:teuthology.orchestra.run.smithi105.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/pacific/81bd20d634209c7cb82c18be12b4b5a05643ebf1/ubuntu/bionic/flavors/default bionic/main amd64 rbd-fuse amd64 16.2.14-548-g81bd20d6-1bionic [78.6 kB] 2024-02-18T01:06:42.149 INFO:teuthology.orchestra.run.smithi155.stdout:Fetched 156 MB in 8s (19.2 MB/s) 2024-02-18T01:06:42.267 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-uritemplate. 2024-02-18T01:06:42.328 INFO:teuthology.orchestra.run.smithi155.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172445 files and directories currently installed.) 2024-02-18T01:06:42.334 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2024-02-18T01:06:42.365 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2024-02-18T01:06:42.626 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjaeger. 2024-02-18T01:06:42.643 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../001-libjaeger_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:42.684 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjaeger (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:42.687 INFO:teuthology.orchestra.run.smithi105.stdout:Fetched 156 MB in 9s (17.9 MB/s) 2024-02-18T01:06:42.855 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-uritemplate. 2024-02-18T01:06:42.912 INFO:teuthology.orchestra.run.smithi105.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172445 files and directories currently installed.) 2024-02-18T01:06:42.917 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../000-python3-uritemplate_0.6-1ubuntu1_all.deb ... 2024-02-18T01:06:42.962 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-uritemplate (0.6-1ubuntu1) ... 2024-02-18T01:06:43.012 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-18T01:06:43.032 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2024-02-18T01:06:43.137 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-18T01:06:43.214 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libjaeger. 2024-02-18T01:06:43.233 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../001-libjaeger_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:43.273 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libjaeger (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:43.440 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../003-librbd1_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:43.541 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librbd1 (16.2.14-548-g81bd20d6-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-18T01:06:43.581 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-02-18T01:06:43.600 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../002-liblttng-ust0_2.10.1-1_amd64.deb ... 2024-02-18T01:06:43.676 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-18T01:06:43.986 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../003-librbd1_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:44.096 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librbd1 (16.2.14-548-g81bd20d6-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-18T01:06:44.232 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../004-librados2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:44.354 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librados2 (16.2.14-548-g81bd20d6-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-18T01:06:44.807 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../004-librados2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:44.934 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librados2 (16.2.14-548-g81bd20d6-1bionic) over (12.2.13-0ubuntu0.18.04.11) ... 2024-02-18T01:06:45.008 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs2. 2024-02-18T01:06:45.027 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../005-libcephfs2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:45.058 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:45.394 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rados. 2024-02-18T01:06:45.410 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../006-python3-rados_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:45.452 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rados (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:45.579 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs2. 2024-02-18T01:06:45.597 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../005-libcephfs2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:45.629 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:45.797 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-18T01:06:45.815 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:45.872 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-argparse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:45.948 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rados. 2024-02-18T01:06:45.967 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../006-python3-rados_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:46.007 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rados (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:46.158 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cephfs. 2024-02-18T01:06:46.177 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../008-python3-cephfs_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:46.217 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cephfs (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:46.351 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-02-18T01:06:46.369 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../007-python3-ceph-argparse_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:46.409 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-argparse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:46.536 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-18T01:06:46.552 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../009-python3-ceph-common_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:46.595 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:46.654 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cephfs. 2024-02-18T01:06:46.669 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../008-python3-cephfs_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:46.879 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cephfs (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:46.939 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-prettytable. 2024-02-18T01:06:46.956 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2024-02-18T01:06:46.998 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2024-02-18T01:06:47.173 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-ceph-common. 2024-02-18T01:06:47.192 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../009-python3-ceph-common_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:47.232 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:47.250 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rbd. 2024-02-18T01:06:47.269 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../011-python3-rbd_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:47.309 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rbd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:47.543 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-prettytable. 2024-02-18T01:06:47.561 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../010-python3-prettytable_0.7.2-3_all.deb ... 2024-02-18T01:06:47.601 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-prettytable (0.7.2-3) ... 2024-02-18T01:06:47.678 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-18T01:06:47.694 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../012-liblua5.3-0_5.3.3-1ubuntu0.18.04.1_amd64.deb ... 2024-02-18T01:06:47.775 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-18T01:06:47.870 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rbd. 2024-02-18T01:06:47.890 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../011-python3-rbd_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:47.930 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rbd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:48.198 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-18T01:06:48.215 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../013-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2024-02-18T01:06:48.232 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-02-18T01:06:48.247 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../012-liblua5.3-0_5.3.3-1ubuntu0.18.04.1_amd64.deb ... 2024-02-18T01:06:48.257 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-18T01:06:48.281 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-18T01:06:48.610 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-18T01:06:48.626 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-02-18T01:06:48.627 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../014-librdkafka1_0.11.3-1build1_amd64.deb ... 2024-02-18T01:06:48.645 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../013-librabbitmq4_0.8.0-1ubuntu0.18.04.2_amd64.deb ... 2024-02-18T01:06:48.668 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-18T01:06:48.684 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-18T01:06:48.988 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package librgw2. 2024-02-18T01:06:49.006 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../015-librgw2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:49.020 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-02-18T01:06:49.038 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking librgw2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:49.038 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../014-librdkafka1_0.11.3-1build1_amd64.deb ... 2024-02-18T01:06:49.079 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-18T01:06:49.415 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package librgw2. 2024-02-18T01:06:49.431 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../015-librgw2_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:49.465 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking librgw2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:49.624 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rgw. 2024-02-18T01:06:49.642 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../016-python3-rgw_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:49.699 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:50.026 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rgw. 2024-02-18T01:06:50.026 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2024-02-18T01:06:50.042 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../016-python3-rgw_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:50.043 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../017-libsnappy1v5_1.1.7-1_amd64.deb ... 2024-02-18T01:06:50.085 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-18T01:06:50.109 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:50.413 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2024-02-18T01:06:50.431 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../018-libleveldb1v5_1.20-2_amd64.deb ... 2024-02-18T01:06:50.432 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libsnappy1v5:amd64. 2024-02-18T01:06:50.450 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../017-libsnappy1v5_1.1.7-1_amd64.deb ... 2024-02-18T01:06:50.463 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2024-02-18T01:06:50.491 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-18T01:06:50.791 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libleveldb1v5:amd64. 2024-02-18T01:06:50.810 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../018-libleveldb1v5_1.20-2_amd64.deb ... 2024-02-18T01:06:50.849 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package liboath0. 2024-02-18T01:06:50.850 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libleveldb1v5:amd64 (1.20-2) ... 2024-02-18T01:06:50.868 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../019-liboath0_2.6.1-1_amd64.deb ... 2024-02-18T01:06:50.907 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking liboath0 (2.6.1-1) ... 2024-02-18T01:06:51.233 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libradosstriper1. 2024-02-18T01:06:51.254 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../020-libradosstriper1_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:51.257 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package liboath0. 2024-02-18T01:06:51.273 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../019-liboath0_2.6.1-1_amd64.deb ... 2024-02-18T01:06:51.316 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking liboath0 (2.6.1-1) ... 2024-02-18T01:06:51.318 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libradosstriper1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:51.619 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libradosstriper1. 2024-02-18T01:06:51.636 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../020-libradosstriper1_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:51.677 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libradosstriper1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:51.758 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-common. 2024-02-18T01:06:51.777 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../021-ceph-common_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:51.818 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:52.136 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-common. 2024-02-18T01:06:52.151 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../021-ceph-common_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:52.193 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:53.787 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-base. 2024-02-18T01:06:53.805 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../022-ceph-base_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:53.855 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-base (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:54.116 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-base. 2024-02-18T01:06:54.132 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../022-ceph-base_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:06:54.184 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-base (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:54.703 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-dateutil. 2024-02-18T01:06:54.719 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../023-python3-dateutil_2.6.1-1_all.deb ... 2024-02-18T01:06:54.758 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2024-02-18T01:06:55.067 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-dateutil. 2024-02-18T01:06:55.073 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-asn1crypto. 2024-02-18T01:06:55.083 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../023-python3-dateutil_2.6.1-1_all.deb ... 2024-02-18T01:06:55.090 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../024-python3-asn1crypto_0.24.0-1_all.deb ... 2024-02-18T01:06:55.125 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-dateutil (2.6.1-1) ... 2024-02-18T01:06:55.149 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2024-02-18T01:06:55.431 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cffi-backend. 2024-02-18T01:06:55.444 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-asn1crypto. 2024-02-18T01:06:55.447 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../025-python3-cffi-backend_1.11.5-1_amd64.deb ... 2024-02-18T01:06:55.460 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../024-python3-asn1crypto_0.24.0-1_all.deb ... 2024-02-18T01:06:55.498 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2024-02-18T01:06:55.503 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-asn1crypto (0.24.0-1) ... 2024-02-18T01:06:55.755 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cffi-backend. 2024-02-18T01:06:55.765 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../025-python3-cffi-backend_1.11.5-1_amd64.deb ... 2024-02-18T01:06:55.805 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cffi-backend (1.11.5-1) ... 2024-02-18T01:06:55.808 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cryptography. 2024-02-18T01:06:55.828 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../026-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2024-02-18T01:06:55.867 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-18T01:06:56.149 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cryptography. 2024-02-18T01:06:56.167 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../026-python3-cryptography_2.1.4-1ubuntu1.4_amd64.deb ... 2024-02-18T01:06:56.199 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-18T01:06:56.262 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-openssl. 2024-02-18T01:06:56.279 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../027-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2024-02-18T01:06:56.321 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2024-02-18T01:06:56.565 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-18T01:06:56.569 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-openssl. 2024-02-18T01:06:56.582 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:56.586 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../027-python3-openssl_17.5.0-1ubuntu1_all.deb ... 2024-02-18T01:06:56.615 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-modules-core (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:56.619 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-openssl (17.5.0-1ubuntu1) ... 2024-02-18T01:06:56.897 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-02-18T01:06:56.970 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:06:57.020 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-18T01:06:57.036 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.4-2_amd64.deb ... 2024-02-18T01:06:57.078 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2024-02-18T01:06:57.180 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-modules-core (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:06:57.397 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-18T01:06:57.415 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-2_all.deb ... 2024-02-18T01:06:57.456 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2024-02-18T01:06:57.551 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-bcrypt. 2024-02-18T01:06:57.567 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.4-2_amd64.deb ... 2024-02-18T01:06:57.609 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-bcrypt (3.1.4-2) ... 2024-02-18T01:06:57.800 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jwt. 2024-02-18T01:06:57.820 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../031-python3-jwt_1.5.3+ds1-1ubuntu0.1_all.deb ... 2024-02-18T01:06:57.859 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-18T01:06:57.871 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cherrypy3. 2024-02-18T01:06:57.889 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-2_all.deb ... 2024-02-18T01:06:57.920 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cherrypy3 (8.9.1-2) ... 2024-02-18T01:06:58.238 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jwt. 2024-02-18T01:06:58.257 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../031-python3-jwt_1.5.3+ds1-1ubuntu0.1_all.deb ... 2024-02-18T01:06:58.322 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-18T01:06:58.404 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-logutils. 2024-02-18T01:06:58.420 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-5_all.deb ... 2024-02-18T01:06:58.462 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-logutils (0.3.3-5) ... 2024-02-18T01:06:58.600 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-logutils. 2024-02-18T01:06:58.620 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../032-python3-logutils_0.3.3-5_all.deb ... 2024-02-18T01:06:58.650 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-logutils (0.3.3-5) ... 2024-02-18T01:06:58.781 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-18T01:06:58.799 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../033-python3-markupsafe_1.0-1build1_amd64.deb ... 2024-02-18T01:06:58.840 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2024-02-18T01:06:58.961 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-markupsafe. 2024-02-18T01:06:58.977 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../033-python3-markupsafe_1.0-1build1_amd64.deb ... 2024-02-18T01:06:59.020 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-markupsafe (1.0-1build1) ... 2024-02-18T01:06:59.158 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-mako. 2024-02-18T01:06:59.177 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../034-python3-mako_1.0.7+ds1-1ubuntu0.2_all.deb ... 2024-02-18T01:06:59.216 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-18T01:06:59.339 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-mako. 2024-02-18T01:06:59.355 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../034-python3-mako_1.0.7+ds1-1ubuntu0.2_all.deb ... 2024-02-18T01:06:59.397 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-18T01:06:59.578 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-18T01:06:59.595 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-1_all.deb ... 2024-02-18T01:06:59.636 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2024-02-18T01:06:59.721 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplegeneric. 2024-02-18T01:06:59.740 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../035-python3-simplegeneric_0.8.1-1_all.deb ... 2024-02-18T01:06:59.780 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplegeneric (0.8.1-1) ... 2024-02-18T01:06:59.897 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-18T01:06:59.917 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2_all.deb ... 2024-02-18T01:06:59.947 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2024-02-18T01:07:00.087 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-singledispatch. 2024-02-18T01:07:00.097 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../036-python3-singledispatch_3.4.0.3-2_all.deb ... 2024-02-18T01:07:00.129 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-singledispatch (3.4.0.3-2) ... 2024-02-18T01:07:00.283 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webob. 2024-02-18T01:07:00.299 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../037-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2024-02-18T01:07:00.341 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2024-02-18T01:07:00.436 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webob. 2024-02-18T01:07:00.452 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../037-python3-webob_1%3a1.7.3-2fakesync1_all.deb ... 2024-02-18T01:07:00.487 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webob (1:1.7.3-2fakesync1) ... 2024-02-18T01:07:00.666 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-bs4. 2024-02-18T01:07:00.682 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../038-python3-bs4_4.6.0-1_all.deb ... 2024-02-18T01:07:00.716 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2024-02-18T01:07:00.806 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-bs4. 2024-02-18T01:07:00.821 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../038-python3-bs4_4.6.0-1_all.deb ... 2024-02-18T01:07:00.856 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-bs4 (4.6.0-1) ... 2024-02-18T01:07:01.041 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-waitress. 2024-02-18T01:07:01.058 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../039-python3-waitress_1.0.1-1_all.deb ... 2024-02-18T01:07:01.125 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-waitress (1.0.1-1) ... 2024-02-18T01:07:01.196 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-waitress. 2024-02-18T01:07:01.211 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../039-python3-waitress_1.0.1-1_all.deb ... 2024-02-18T01:07:01.254 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-waitress (1.0.1-1) ... 2024-02-18T01:07:01.467 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../040-python3-pkg-resources_39.0.1-2ubuntu0.1_all.deb ... 2024-02-18T01:07:01.599 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../040-python3-pkg-resources_39.0.1-2ubuntu0.1_all.deb ... 2024-02-18T01:07:01.744 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pkg-resources (39.0.1-2ubuntu0.1) over (39.0.1-2) ... 2024-02-18T01:07:01.859 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pkg-resources (39.0.1-2ubuntu0.1) over (39.0.1-2) ... 2024-02-18T01:07:02.102 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tempita. 2024-02-18T01:07:02.121 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2024-02-18T01:07:02.160 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tempita (0.5.2-2) ... 2024-02-18T01:07:02.238 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tempita. 2024-02-18T01:07:02.255 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../041-python3-tempita_0.5.2-2_all.deb ... 2024-02-18T01:07:02.296 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tempita (0.5.2-2) ... 2024-02-18T01:07:02.480 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-paste. 2024-02-18T01:07:02.496 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2024-02-18T01:07:02.538 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-18T01:07:02.632 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-paste. 2024-02-18T01:07:02.647 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../042-python3-paste_2.0.3+dfsg-4ubuntu1_all.deb ... 2024-02-18T01:07:02.690 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-18T01:07:02.883 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-18T01:07:02.902 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2024-02-18T01:07:02.941 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2024-02-18T01:07:03.010 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-02-18T01:07:03.027 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../043-python-pastedeploy-tpl_1.5.2-4_all.deb ... 2024-02-18T01:07:03.069 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-pastedeploy-tpl (1.5.2-4) ... 2024-02-18T01:07:03.396 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-18T01:07:03.412 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2024-02-18T01:07:03.454 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2024-02-18T01:07:03.707 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webtest. 2024-02-18T01:07:03.723 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2024-02-18T01:07:03.757 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2024-02-18T01:07:04.091 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pecan. 2024-02-18T01:07:04.094 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastedeploy. 2024-02-18T01:07:04.108 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2024-02-18T01:07:04.113 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../044-python3-pastedeploy_1.5.2-4_all.deb ... 2024-02-18T01:07:04.142 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pecan (1.2.1-2) ... 2024-02-18T01:07:04.152 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastedeploy (1.5.2-4) ... 2024-02-18T01:07:04.464 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webtest. 2024-02-18T01:07:04.480 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../045-python3-webtest_2.0.28-1ubuntu1_all.deb ... 2024-02-18T01:07:04.497 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-18T01:07:04.517 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.2_all.deb ... 2024-02-18T01:07:04.522 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webtest (2.0.28-1ubuntu1) ... 2024-02-18T01:07:04.555 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-18T01:07:04.858 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pecan. 2024-02-18T01:07:04.877 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../046-python3-pecan_1.2.1-2_all.deb ... 2024-02-18T01:07:04.917 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-18T01:07:04.917 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pecan (1.2.1-2) ... 2024-02-18T01:07:04.932 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:04.975 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:05.297 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-werkzeug. 2024-02-18T01:07:05.311 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr. 2024-02-18T01:07:05.313 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../047-python3-werkzeug_0.14.1+dfsg1-1ubuntu0.2_all.deb ... 2024-02-18T01:07:05.329 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../049-ceph-mgr_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:05.355 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-18T01:07:05.369 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:05.715 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-02-18T01:07:05.733 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../048-libsqlite3-mod-ceph_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:05.774 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libsqlite3-mod-ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:05.839 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mon. 2024-02-18T01:07:05.858 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../050-ceph-mon_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:05.889 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mon (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:06.102 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr. 2024-02-18T01:07:06.121 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../049-ceph-mgr_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:06.160 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:06.613 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mon. 2024-02-18T01:07:06.631 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../050-ceph-mon_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:06.642 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-osd. 2024-02-18T01:07:06.659 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../051-ceph-osd_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:06.663 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mon (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:06.701 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-osd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:07.434 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-osd. 2024-02-18T01:07:07.451 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../051-ceph-osd_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:07.492 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-osd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:08.648 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph. 2024-02-18T01:07:08.664 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../052-ceph_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:08.706 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:09.050 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-fuse. 2024-02-18T01:07:09.067 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../053-ceph-fuse_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:09.108 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:09.122 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph. 2024-02-18T01:07:09.141 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../052-ceph_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:09.181 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:09.508 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-fuse. 2024-02-18T01:07:09.526 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../053-ceph-fuse_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:09.536 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mds. 2024-02-18T01:07:09.556 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../054-ceph-mds_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:09.566 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:09.620 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mds (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:10.011 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mds. 2024-02-18T01:07:10.023 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../054-ceph-mds_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:10.061 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mds (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:10.132 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package cephadm. 2024-02-18T01:07:10.147 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../055-cephadm_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:10.190 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:10.517 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-jinja2. 2024-02-18T01:07:10.536 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2024-02-18T01:07:10.573 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package cephadm. 2024-02-18T01:07:10.575 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-18T01:07:10.592 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../055-cephadm_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:10.623 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:10.895 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-18T01:07:10.911 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:10.922 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-jinja2. 2024-02-18T01:07:10.937 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../056-python3-jinja2_2.10-1ubuntu0.18.04.1_all.deb ... 2024-02-18T01:07:10.953 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:10.975 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-18T01:07:11.298 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-18T01:07:11.315 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2024-02-18T01:07:11.332 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-02-18T01:07:11.344 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../057-ceph-mgr-cephadm_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:11.356 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2024-02-18T01:07:11.387 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:11.642 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-routes. 2024-02-18T01:07:11.662 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2024-02-18T01:07:11.700 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-routes (2.4.1-1) ... 2024-02-18T01:07:11.765 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-repoze.lru. 2024-02-18T01:07:11.782 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../058-python3-repoze.lru_0.7-1_all.deb ... 2024-02-18T01:07:11.823 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-repoze.lru (0.7-1) ... 2024-02-18T01:07:12.011 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-18T01:07:12.030 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:12.069 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-dashboard (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:12.126 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-routes. 2024-02-18T01:07:12.142 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../059-python3-routes_2.4.1-1_all.deb ... 2024-02-18T01:07:12.201 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-routes (2.4.1-1) ... 2024-02-18T01:07:12.512 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-02-18T01:07:12.532 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../060-ceph-mgr-dashboard_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:12.563 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-dashboard (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:13.051 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-numpy. 2024-02-18T01:07:13.071 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2024-02-18T01:07:13.109 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-18T01:07:13.588 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-numpy. 2024-02-18T01:07:13.608 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../061-python3-numpy_1%3a1.13.3-2ubuntu1_amd64.deb ... 2024-02-18T01:07:13.655 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-18T01:07:13.728 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-decorator. 2024-02-18T01:07:13.747 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2024-02-18T01:07:13.780 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-decorator (4.1.2-1) ... 2024-02-18T01:07:14.083 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-scipy. 2024-02-18T01:07:14.100 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2024-02-18T01:07:14.142 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2024-02-18T01:07:14.275 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-decorator. 2024-02-18T01:07:14.292 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../062-python3-decorator_4.1.2-1_all.deb ... 2024-02-18T01:07:14.334 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-decorator (4.1.2-1) ... 2024-02-18T01:07:14.670 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-scipy. 2024-02-18T01:07:14.687 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../063-python3-scipy_0.19.1-2ubuntu1_amd64.deb ... 2024-02-18T01:07:14.729 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-scipy (0.19.1-2ubuntu1) ... 2024-02-18T01:07:15.387 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-18T01:07:15.405 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2024-02-18T01:07:15.445 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2024-02-18T01:07:15.901 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-joblib. 2024-02-18T01:07:15.919 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2024-02-18T01:07:15.938 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-02-18T01:07:15.957 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../064-python3-sklearn-lib_0.19.1-3_amd64.deb ... 2024-02-18T01:07:15.959 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-joblib (0.11-1) ... 2024-02-18T01:07:15.996 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn-lib (0.19.1-3) ... 2024-02-18T01:07:16.320 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sklearn. 2024-02-18T01:07:16.331 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2024-02-18T01:07:16.362 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2024-02-18T01:07:16.465 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-joblib. 2024-02-18T01:07:16.483 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../065-python3-joblib_0.11-1_all.deb ... 2024-02-18T01:07:16.525 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-joblib (0.11-1) ... 2024-02-18T01:07:16.905 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sklearn. 2024-02-18T01:07:16.923 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../066-python3-sklearn_0.19.1-3_all.deb ... 2024-02-18T01:07:16.943 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-18T01:07:16.960 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:16.964 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sklearn (0.19.1-3) ... 2024-02-18T01:07:17.002 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:17.564 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-02-18T01:07:17.585 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../067-ceph-mgr-diskprediction-local_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:17.623 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-diskprediction-local (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:18.188 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-monotonic. 2024-02-18T01:07:18.204 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2024-02-18T01:07:18.246 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-monotonic (1.1-2) ... 2024-02-18T01:07:18.548 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-fasteners. 2024-02-18T01:07:18.566 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2024-02-18T01:07:18.624 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2024-02-18T01:07:18.768 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-monotonic. 2024-02-18T01:07:18.784 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../068-python3-monotonic_1.1-2_all.deb ... 2024-02-18T01:07:18.826 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-monotonic (1.1-2) ... 2024-02-18T01:07:18.977 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-httplib2. 2024-02-18T01:07:18.995 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2024-02-18T01:07:19.035 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-18T01:07:19.137 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-fasteners. 2024-02-18T01:07:19.155 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../069-python3-fasteners_0.12.0-3_all.deb ... 2024-02-18T01:07:19.195 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-fasteners (0.12.0-3) ... 2024-02-18T01:07:19.321 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-18T01:07:19.338 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2024-02-18T01:07:19.379 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2024-02-18T01:07:19.481 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-httplib2. 2024-02-18T01:07:19.500 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../070-python3-httplib2_0.9.2+dfsg-1ubuntu0.3_all.deb ... 2024-02-18T01:07:19.564 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-18T01:07:19.682 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-18T01:07:19.699 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2024-02-18T01:07:19.732 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-18T01:07:19.875 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1. 2024-02-18T01:07:19.894 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../071-python3-pyasn1_0.4.2-3_all.deb ... 2024-02-18T01:07:19.934 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1 (0.4.2-3) ... 2024-02-18T01:07:19.993 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-rsa. 2024-02-18T01:07:20.009 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2024-02-18T01:07:20.085 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-rsa (3.4.2-1) ... 2024-02-18T01:07:20.303 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-02-18T01:07:20.320 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../072-python3-pyasn1-modules_0.2.1-0.2_all.deb ... 2024-02-18T01:07:20.361 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-18T01:07:20.396 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-sqlalchemy. 2024-02-18T01:07:20.413 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2024-02-18T01:07:20.454 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-18T01:07:20.706 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-rsa. 2024-02-18T01:07:20.723 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../073-python3-rsa_3.4.2-1_all.deb ... 2024-02-18T01:07:20.764 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-rsa (3.4.2-1) ... 2024-02-18T01:07:20.925 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-oauth2client. 2024-02-18T01:07:20.942 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2024-02-18T01:07:20.992 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2024-02-18T01:07:21.059 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-sqlalchemy. 2024-02-18T01:07:21.076 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../074-python3-sqlalchemy_1.1.11+ds1-1ubuntu1_all.deb ... 2024-02-18T01:07:21.117 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-18T01:07:21.328 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-setuptools. 2024-02-18T01:07:21.347 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2ubuntu0.1_all.deb ... 2024-02-18T01:07:21.386 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:21.521 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-oauth2client. 2024-02-18T01:07:21.539 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../075-python3-oauth2client_4.1.2-3_all.deb ... 2024-02-18T01:07:21.571 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-oauth2client (4.1.2-3) ... 2024-02-18T01:07:21.798 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-websocket. 2024-02-18T01:07:21.876 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2024-02-18T01:07:21.876 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2024-02-18T01:07:21.907 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-setuptools. 2024-02-18T01:07:21.924 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../076-python3-setuptools_39.0.1-2ubuntu0.1_all.deb ... 2024-02-18T01:07:21.965 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:22.168 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-18T01:07:22.184 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2024-02-18T01:07:22.226 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-18T01:07:22.344 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-websocket. 2024-02-18T01:07:22.362 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../077-python3-websocket_0.44.0-0ubuntu2_all.deb ... 2024-02-18T01:07:22.393 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-websocket (0.44.0-0ubuntu2) ... 2024-02-18T01:07:22.630 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-18T01:07:22.647 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:22.705 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-kubernetes. 2024-02-18T01:07:22.725 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../078-python3-kubernetes_2.0.0-2ubuntu1_all.deb ... 2024-02-18T01:07:22.763 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-18T01:07:22.766 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:23.094 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libonig4:amd64. 2024-02-18T01:07:23.111 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2024-02-18T01:07:23.152 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2024-02-18T01:07:23.209 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-02-18T01:07:23.225 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../079-ceph-mgr-k8sevents_16.2.14-548-g81bd20d6-1bionic_all.deb ... 2024-02-18T01:07:23.376 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-mgr-k8sevents (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:23.480 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-18T01:07:23.498 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2024-02-18T01:07:23.530 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2024-02-18T01:07:23.680 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libonig4:amd64. 2024-02-18T01:07:23.701 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../080-libonig4_6.7.0-1_amd64.deb ... 2024-02-18T01:07:23.739 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libonig4:amd64 (6.7.0-1) ... 2024-02-18T01:07:23.807 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package jq. 2024-02-18T01:07:23.825 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2024-02-18T01:07:23.865 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking jq (1.5+dfsg-2) ... 2024-02-18T01:07:24.092 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libjq1:amd64. 2024-02-18T01:07:24.108 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../081-libjq1_1.5+dfsg-2_amd64.deb ... 2024-02-18T01:07:24.135 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package socat. 2024-02-18T01:07:24.152 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2024-02-18T01:07:24.158 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libjq1:amd64 (1.5+dfsg-2) ... 2024-02-18T01:07:24.193 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2024-02-18T01:07:24.403 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package jq. 2024-02-18T01:07:24.420 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../082-jq_1.5+dfsg-2_amd64.deb ... 2024-02-18T01:07:24.486 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking jq (1.5+dfsg-2) ... 2024-02-18T01:07:24.572 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libxslt1.1:amd64. 2024-02-18T01:07:24.589 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.3_amd64.deb ... 2024-02-18T01:07:24.630 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-18T01:07:24.747 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package socat. 2024-02-18T01:07:24.766 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../083-socat_1.7.3.2-2ubuntu2_amd64.deb ... 2024-02-18T01:07:24.805 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking socat (1.7.3.2-2ubuntu2) ... 2024-02-18T01:07:24.897 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package xmlstarlet. 2024-02-18T01:07:24.915 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2024-02-18T01:07:24.958 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2024-02-18T01:07:25.184 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libxslt1.1:amd64. 2024-02-18T01:07:25.203 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../084-libxslt1.1_1.1.29-5ubuntu0.3_amd64.deb ... 2024-02-18T01:07:25.243 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-18T01:07:25.264 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package ceph-test. 2024-02-18T01:07:25.281 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../086-ceph-test_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:25.322 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking ceph-test (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:25.512 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package xmlstarlet. 2024-02-18T01:07:25.529 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../085-xmlstarlet_1.6.1-2_amd64.deb ... 2024-02-18T01:07:25.604 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking xmlstarlet (1.6.1-2) ... 2024-02-18T01:07:25.959 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package ceph-test. 2024-02-18T01:07:25.978 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../086-ceph-test_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:26.026 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking ceph-test (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:28.600 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-18T01:07:28.621 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../087-libcephfs-dev_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:28.658 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking libcephfs-dev (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:28.928 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package nvme-cli. 2024-02-18T01:07:28.945 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.2_amd64.deb ... 2024-02-18T01:07:28.986 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking nvme-cli (1.5-1ubuntu1.2) ... 2024-02-18T01:07:29.340 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-attr. 2024-02-18T01:07:29.358 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2024-02-18T01:07:29.398 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-attr (17.4.0-2) ... 2024-02-18T01:07:29.412 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package libcephfs-dev. 2024-02-18T01:07:29.430 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../087-libcephfs-dev_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:29.487 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking libcephfs-dev (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:29.676 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-funcsigs. 2024-02-18T01:07:29.692 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2024-02-18T01:07:29.734 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2024-02-18T01:07:29.739 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package nvme-cli. 2024-02-18T01:07:29.758 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../088-nvme-cli_1.5-1ubuntu1.2_amd64.deb ... 2024-02-18T01:07:29.789 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking nvme-cli (1.5-1ubuntu1.2) ... 2024-02-18T01:07:30.037 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-pluggy. 2024-02-18T01:07:30.055 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2024-02-18T01:07:30.095 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-pluggy (0.6.0-1) ... 2024-02-18T01:07:30.152 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-attr. 2024-02-18T01:07:30.168 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../089-python-attr_17.4.0-2_all.deb ... 2024-02-18T01:07:30.210 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-attr (17.4.0-2) ... 2024-02-18T01:07:30.398 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-py. 2024-02-18T01:07:30.415 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../092-python-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-18T01:07:30.456 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:30.513 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-funcsigs. 2024-02-18T01:07:30.531 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../090-python-funcsigs_1.0.2-4_all.deb ... 2024-02-18T01:07:30.604 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-funcsigs (1.0.2-4) ... 2024-02-18T01:07:30.801 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python-pytest. 2024-02-18T01:07:30.819 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2024-02-18T01:07:30.859 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python-pytest (3.3.2-2) ... 2024-02-18T01:07:30.907 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pluggy. 2024-02-18T01:07:30.923 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../091-python-pluggy_0.6.0-1_all.deb ... 2024-02-18T01:07:30.965 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pluggy (0.6.0-1) ... 2024-02-18T01:07:31.204 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-attr. 2024-02-18T01:07:31.220 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2024-02-18T01:07:31.262 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-attr (17.4.0-2) ... 2024-02-18T01:07:31.301 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-py. 2024-02-18T01:07:31.319 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../092-python-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-18T01:07:31.359 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:31.573 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-cycler. 2024-02-18T01:07:31.594 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2024-02-18T01:07:31.631 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-cycler (0.10.0-1) ... 2024-02-18T01:07:31.696 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python-pytest. 2024-02-18T01:07:31.712 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../093-python-pytest_3.3.2-2_all.deb ... 2024-02-18T01:07:31.754 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python-pytest (3.3.2-2) ... 2024-02-18T01:07:31.901 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-webencodings. 2024-02-18T01:07:31.921 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2024-02-18T01:07:31.967 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-webencodings (0.5-2) ... 2024-02-18T01:07:32.073 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-attr. 2024-02-18T01:07:32.094 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../094-python3-attr_17.4.0-2_all.deb ... 2024-02-18T01:07:32.132 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-attr (17.4.0-2) ... 2024-02-18T01:07:32.278 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-html5lib. 2024-02-18T01:07:32.296 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2024-02-18T01:07:32.336 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2024-02-18T01:07:32.426 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-cycler. 2024-02-18T01:07:32.446 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../095-python3-cycler_0.10.0-1_all.deb ... 2024-02-18T01:07:32.484 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-cycler (0.10.0-1) ... 2024-02-18T01:07:32.673 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-18T01:07:32.691 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.6_amd64.deb ... 2024-02-18T01:07:32.731 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-18T01:07:32.812 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-webencodings. 2024-02-18T01:07:32.829 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../096-python3-webencodings_0.5-2_all.deb ... 2024-02-18T01:07:32.870 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-webencodings (0.5-2) ... 2024-02-18T01:07:33.174 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-html5lib. 2024-02-18T01:07:33.184 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyparsing. 2024-02-18T01:07:33.191 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../097-python3-html5lib_0.999999999-1_all.deb ... 2024-02-18T01:07:33.203 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2024-02-18T01:07:33.231 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-html5lib (0.999999999-1) ... 2024-02-18T01:07:33.243 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-18T01:07:33.545 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tz. 2024-02-18T01:07:33.565 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2024-02-18T01:07:33.603 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tz (2018.3-2) ... 2024-02-18T01:07:33.609 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-02-18T01:07:33.626 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../098-python3-lxml_4.2.1-1ubuntu0.6_amd64.deb ... 2024-02-18T01:07:33.667 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-18T01:07:33.939 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-matplotlib. 2024-02-18T01:07:33.956 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2024-02-18T01:07:33.998 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-18T01:07:34.120 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyparsing. 2024-02-18T01:07:34.137 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../099-python3-pyparsing_2.2.0+dfsg1-2_all.deb ... 2024-02-18T01:07:34.179 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-18T01:07:34.515 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tz. 2024-02-18T01:07:34.534 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../100-python3-tz_2018.3-2_all.deb ... 2024-02-18T01:07:34.573 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tz (2018.3-2) ... 2024-02-18T01:07:34.653 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-olefile. 2024-02-18T01:07:34.673 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2024-02-18T01:07:34.720 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-olefile (0.45.1-1) ... 2024-02-18T01:07:34.875 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-matplotlib. 2024-02-18T01:07:34.892 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../101-python3-matplotlib_2.1.1-2ubuntu3_amd64.deb ... 2024-02-18T01:07:34.964 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-18T01:07:35.006 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pastescript. 2024-02-18T01:07:35.023 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2024-02-18T01:07:35.064 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2024-02-18T01:07:35.384 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pil:amd64. 2024-02-18T01:07:35.401 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.8_amd64.deb ... 2024-02-18T01:07:35.467 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-18T01:07:35.672 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-olefile. 2024-02-18T01:07:35.689 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../102-python3-olefile_0.45.1-1_all.deb ... 2024-02-18T01:07:35.730 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-olefile (0.45.1-1) ... 2024-02-18T01:07:35.821 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pluggy. 2024-02-18T01:07:35.839 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2024-02-18T01:07:35.887 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2024-02-18T01:07:36.000 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pastescript. 2024-02-18T01:07:36.016 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-2_all.deb ... 2024-02-18T01:07:36.091 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pastescript (2.0.2-2) ... 2024-02-18T01:07:36.173 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-py. 2024-02-18T01:07:36.191 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../106-python3-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-18T01:07:36.250 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:36.412 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pil:amd64. 2024-02-18T01:07:36.428 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../104-python3-pil_5.1.0-1ubuntu0.8_amd64.deb ... 2024-02-18T01:07:36.470 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-18T01:07:36.576 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-18T01:07:36.595 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2024-02-18T01:07:36.643 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2024-02-18T01:07:36.864 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pluggy. 2024-02-18T01:07:36.883 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../105-python3-pluggy_0.6.0-1_all.deb ... 2024-02-18T01:07:36.912 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-pytest. 2024-02-18T01:07:36.923 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pluggy (0.6.0-1) ... 2024-02-18T01:07:36.931 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2024-02-18T01:07:36.962 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-pytest (3.3.2-2) ... 2024-02-18T01:07:37.242 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-py. 2024-02-18T01:07:37.259 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../106-python3-py_1.5.2-1ubuntu0.1_all.deb ... 2024-02-18T01:07:37.300 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:37.340 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-simplejson. 2024-02-18T01:07:37.357 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2024-02-18T01:07:37.390 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2024-02-18T01:07:37.620 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pyinotify. 2024-02-18T01:07:37.637 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../107-python3-pyinotify_0.9.6-1_all.deb ... 2024-02-18T01:07:37.678 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pyinotify (0.9.6-1) ... 2024-02-18T01:07:37.701 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package python3-tk:amd64. 2024-02-18T01:07:37.719 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2024-02-18T01:07:37.751 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-18T01:07:37.997 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-pytest. 2024-02-18T01:07:38.017 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../108-python3-pytest_3.3.2-2_all.deb ... 2024-02-18T01:07:38.055 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-pytest (3.3.2-2) ... 2024-02-18T01:07:38.213 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package radosgw. 2024-02-18T01:07:38.233 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../111-radosgw_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:38.271 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking radosgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:38.392 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-simplejson. 2024-02-18T01:07:38.411 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../109-python3-simplejson_3.13.2-1_amd64.deb ... 2024-02-18T01:07:38.450 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-simplejson (3.13.2-1) ... 2024-02-18T01:07:38.753 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package python3-tk:amd64. 2024-02-18T01:07:38.773 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../110-python3-tk_3.6.9-1~18.04_amd64.deb ... 2024-02-18T01:07:38.845 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-18T01:07:39.281 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package radosgw. 2024-02-18T01:07:39.299 INFO:teuthology.orchestra.run.smithi105.stdout:Selecting previously unselected package rbd-fuse. 2024-02-18T01:07:39.301 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../111-radosgw_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:39.317 INFO:teuthology.orchestra.run.smithi105.stdout:Preparing to unpack .../112-rbd-fuse_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:39.340 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking radosgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:39.357 INFO:teuthology.orchestra.run.smithi105.stdout:Unpacking rbd-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:39.681 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-logutils (0.3.3-5) ... 2024-02-18T01:07:40.013 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-18T01:07:40.368 INFO:teuthology.orchestra.run.smithi155.stdout:Selecting previously unselected package rbd-fuse. 2024-02-18T01:07:40.385 INFO:teuthology.orchestra.run.smithi155.stdout:Preparing to unpack .../112-rbd-fuse_16.2.14-548-g81bd20d6-1bionic_amd64.deb ... 2024-02-18T01:07:40.398 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-18T01:07:40.426 INFO:teuthology.orchestra.run.smithi155.stdout:Unpacking rbd-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:40.524 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-18T01:07:40.714 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-logutils (0.3.3-5) ... 2024-02-18T01:07:41.048 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-werkzeug (0.14.1+dfsg1-1ubuntu0.2) ... 2024-02-18T01:07:41.152 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-18T01:07:41.424 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librdkafka1:amd64 (0.11.3-1build1) ... 2024-02-18T01:07:41.520 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2024-02-18T01:07:41.575 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sqlalchemy (1.1.11+ds1-1ubuntu1) ... 2024-02-18T01:07:41.805 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webencodings (0.5-2) ... 2024-02-18T01:07:42.139 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-attr (17.4.0-2) ... 2024-02-18T01:07:42.187 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pil:amd64 (5.1.0-1ubuntu0.8) ... 2024-02-18T01:07:42.340 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2024-02-18T01:07:42.466 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-waitress (1.0.1-1) ... 2024-02-18T01:07:42.563 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplegeneric (0.8.1-1) ... 2024-02-18T01:07:42.800 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2024-02-18T01:07:42.874 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webencodings (0.5-2) ... 2024-02-18T01:07:43.119 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-olefile (0.45.1-1) ... 2024-02-18T01:07:43.174 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-attr (17.4.0-2) ... 2024-02-18T01:07:43.400 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cffi-backend (1.11.5-1) ... 2024-02-18T01:07:43.412 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-argparse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:43.518 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-waitress (1.0.1-1) ... 2024-02-18T01:07:43.729 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tempita (0.5.2-2) ... 2024-02-18T01:07:43.899 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/waitress-serve-python3 to provide /usr/bin/waitress-serve (waitress-serve) in auto mode 2024-02-18T01:07:44.039 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2024-02-18T01:07:44.165 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-18T01:07:44.220 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-olefile (0.45.1-1) ... 2024-02-18T01:07:44.521 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-argparse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:44.831 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tempita (0.5.2-2) ... 2024-02-18T01:07:45.134 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2024-02-18T01:07:45.157 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up socat (1.7.3.2-2ubuntu2) ... 2024-02-18T01:07:45.283 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-numpy (1:1.13.3-2ubuntu1) ... 2024-02-18T01:07:45.436 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2024-02-18T01:07:45.703 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:45.932 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-simplejson (3.13.2-1) ... 2024-02-18T01:07:46.202 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webob (1:1.7.3-2fakesync1) ... 2024-02-18T01:07:46.264 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2024-02-18T01:07:46.390 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-pluggy (0.6.0-1) ... 2024-02-18T01:07:46.545 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bcrypt (3.1.4-2) ... 2024-02-18T01:07:46.591 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2024-02-18T01:07:46.838 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:07:46.917 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-18T01:07:47.089 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-simplejson (3.13.2-1) ... 2024-02-18T01:07:47.243 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cycler (0.10.0-1) ... 2024-02-18T01:07:47.433 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libonig4:amd64 (6.7.0-1) ... 2024-02-18T01:07:47.528 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2024-02-18T01:07:47.558 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pluggy (0.6.0-1) ... 2024-02-18T01:07:47.792 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-singledispatch (3.4.0.3-2) ... 2024-02-18T01:07:47.879 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pkg-resources (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:48.068 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyparsing (2.2.0+dfsg1-2) ... 2024-02-18T01:07:48.222 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-bs4 (4.6.0-1) ... 2024-02-18T01:07:48.353 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cycler (0.10.0-1) ... 2024-02-18T01:07:48.573 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pluggy (0.6.0-1) ... 2024-02-18T01:07:48.637 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cherrypy3 (8.9.1-2) ... 2024-02-18T01:07:48.866 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:49.013 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pkg-resources (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:49.356 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-bs4 (4.6.0-1) ... 2024-02-18T01:07:49.509 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user cephadm....done 2024-02-18T01:07:49.682 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pluggy (0.6.0-1) ... 2024-02-18T01:07:49.689 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2024-02-18T01:07:49.967 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:49.990 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-18T01:07:50.116 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up nvme-cli (1.5-1ubuntu1.2) ... 2024-02-18T01:07:50.258 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2024-02-18T01:07:50.584 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user cephadm....done 2024-02-18T01:07:50.601 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-prettytable (0.7.2-3) ... 2024-02-18T01:07:50.748 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-markupsafe (1.0-1build1) ... 2024-02-18T01:07:50.911 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2024-02-18T01:07:51.029 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2024-02-18T01:07:51.049 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libxslt1.1:amd64 (1.1.29-5ubuntu0.3) ... 2024-02-18T01:07:51.175 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up nvme-cli (1.5-1ubuntu1.2) ... 2024-02-18T01:07:51.318 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-asn1crypto (0.24.0-1) ... 2024-02-18T01:07:51.355 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:51.661 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-prettytable (0.7.2-3) ... 2024-02-18T01:07:51.673 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-monotonic (1.1-2) ... 2024-02-18T01:07:51.945 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjq1:amd64 (1.5+dfsg-2) ... 2024-02-18T01:07:51.974 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-18T01:07:52.089 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1 (0.4.2-3) ... 2024-02-18T01:07:52.100 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-18T01:07:52.431 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:52.459 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-html5lib (0.999999999-1) ... 2024-02-18T01:07:52.757 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-monotonic (1.1-2) ... 2024-02-18T01:07:52.819 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-repoze.lru (0.7-1) ... 2024-02-18T01:07:53.058 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libsnappy1v5:amd64 (1.1.7-1) ... 2024-02-18T01:07:53.129 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-18T01:07:53.209 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jinja2 (2.10-1ubuntu0.18.04.1) ... 2024-02-18T01:07:53.413 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-18T01:07:53.560 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-html5lib (0.999999999-1) ... 2024-02-18T01:07:53.764 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-attr (17.4.0-2) ... 2024-02-18T01:07:53.895 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-repoze.lru (0.7-1) ... 2024-02-18T01:07:54.065 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-funcsigs (1.0.2-4) ... 2024-02-18T01:07:54.179 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-httplib2 (0.9.2+dfsg-1ubuntu0.3) ... 2024-02-18T01:07:54.266 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libjaeger (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:54.392 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-18T01:07:54.497 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2) ... 2024-02-18T01:07:54.811 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2024-02-18T01:07:54.848 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-attr (17.4.0-2) ... 2024-02-18T01:07:54.899 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-18T01:07:55.129 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liboath0 (2.6.1-1) ... 2024-02-18T01:07:55.157 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-funcsigs (1.0.2-4) ... 2024-02-18T01:07:55.289 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-18T01:07:55.375 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libjaeger (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:07:55.414 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-18T01:07:55.501 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-paste (2.0.3+dfsg-4ubuntu1) ... 2024-02-18T01:07:55.691 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2024-02-18T01:07:55.816 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:55.919 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-websocket (0.44.0-0ubuntu2) ... 2024-02-18T01:07:55.992 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-02-18T01:07:56.218 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-18T01:07:56.246 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liboath0 (2.6.1-1) ... 2024-02-18T01:07:56.369 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-18T01:07:56.447 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librabbitmq4:amd64 (0.8.0-1ubuntu0.18.04.2) ... 2024-02-18T01:07:56.495 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-decorator (4.1.2-1) ... 2024-02-18T01:07:56.573 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tk:amd64 (3.6.9-1~18.04) ... 2024-02-18T01:07:56.771 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-tz (2018.3-2) ... 2024-02-18T01:07:56.824 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pastedeploy-tpl (1.5.2-4) ... 2024-02-18T01:07:56.971 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-setuptools (39.0.1-2ubuntu0.1) ... 2024-02-18T01:07:57.056 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2024-02-18T01:07:57.360 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblttng-ust0:amd64 (2.10.1-1) ... 2024-02-18T01:07:57.368 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-18T01:07:57.486 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1ubuntu0.18.04.1) ... 2024-02-18T01:07:57.612 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-decorator (4.1.2-1) ... 2024-02-18T01:07:57.675 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-joblib (0.11-1) ... 2024-02-18T01:07:57.896 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-tz (2018.3-2) ... 2024-02-18T01:07:58.042 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-routes (2.4.1-1) ... 2024-02-18T01:07:58.189 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-uritemplate (0.6-1ubuntu1) ... 2024-02-18T01:07:58.360 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2024-02-18T01:07:58.490 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-jwt (1.5.3+ds1-1ubuntu0.1) ... 2024-02-18T01:07:58.670 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-dateutil (2.6.1-1) ... 2024-02-18T01:07:58.799 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-joblib (0.11-1) ... 2024-02-18T01:07:59.004 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-18T01:07:59.193 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-routes (2.4.1-1) ... 2024-02-18T01:07:59.356 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-fasteners (0.12.0-3) ... 2024-02-18T01:07:59.649 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up jq (1.5+dfsg-2) ... 2024-02-18T01:07:59.744 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pyinotify (0.9.6-1) ... 2024-02-18T01:07:59.775 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2024-02-18T01:08:00.044 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-dateutil (2.6.1-1) ... 2024-02-18T01:08:00.337 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cryptography (2.1.4-1ubuntu1.4) ... 2024-02-18T01:08:00.681 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-fasteners (0.12.0-3) ... 2024-02-18T01:08:00.938 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:08:00.973 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up jq (1.5+dfsg-2) ... 2024-02-18T01:08:01.116 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-scipy (0.19.1-2ubuntu1) ... 2024-02-18T01:08:01.264 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-18T01:08:01.582 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-18T01:08:01.933 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2024-02-18T01:08:02.059 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pytest (3.3.2-2) ... 2024-02-18T01:08:02.288 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-py (1.5.2-1ubuntu0.1) ... 2024-02-18T01:08:02.444 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python-pytest (3.3.2-2) ... 2024-02-18T01:08:02.597 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-lxml:amd64 (4.2.1-1ubuntu0.6) ... 2024-02-18T01:08:02.737 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2024-02-18T01:08:02.855 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up xmlstarlet (1.6.1-2) ... 2024-02-18T01:08:02.924 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-mako (1.0.7+ds1-1ubuntu0.2) ... 2024-02-18T01:08:02.956 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librados2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:03.082 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rsa (3.4.2-1) ... 2024-02-18T01:08:03.275 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn-lib (0.19.1-3) ... 2024-02-18T01:08:03.399 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:03.403 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pytest (3.3.2-2) ... 2024-02-18T01:08:03.779 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python-pytest (3.3.2-2) ... 2024-02-18T01:08:03.850 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libcephfs-dev (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:03.995 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rados (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:04.030 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libleveldb1v5:amd64 (1.20-2) ... 2024-02-18T01:08:04.121 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2024-02-18T01:08:04.181 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up xmlstarlet (1.6.1-2) ... 2024-02-18T01:08:04.307 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librados2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:04.433 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rsa (3.4.2-1) ... 2024-02-18T01:08:04.439 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2024-02-18T01:08:04.732 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2024-02-18T01:08:04.742 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:04.868 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libcephfs-dev (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:04.986 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rados (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:05.112 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-openssl (17.5.0-1ubuntu1) ... 2024-02-18T01:08:05.151 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-18T01:08:05.446 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastedeploy (1.5.2-4) ... 2024-02-18T01:08:05.747 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-oauth2client (4.1.2-3) ... 2024-02-18T01:08:06.055 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-sklearn (0.19.1-3) ... 2024-02-18T01:08:06.159 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-matplotlib (2.1.1-2ubuntu3) ... 2024-02-18T01:08:06.893 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2024-02-18T01:08:07.021 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-sklearn (0.19.1-3) ... 2024-02-18T01:08:07.186 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libradosstriper1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:07.320 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-cephfs (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:07.638 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up libsqlite3-mod-ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:07.739 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librgw2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:07.850 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-webtest (2.0.28-1ubuntu1) ... 2024-02-18T01:08:07.865 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.080 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-18T01:08:08.081 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-18T01:08:08.159 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libradosstriper1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.285 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-cephfs (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.358 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up librbd1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.509 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pecan (1.2.1-2) ... 2024-02-18T01:08:08.595 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up libsqlite3-mod-ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.599 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2024-02-18T01:08:08.631 INFO:teuthology.orchestra.run.smithi105.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2024-02-18T01:08:08.721 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librgw2 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.830 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:08.954 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-18T01:08:09.026 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-18T01:08:09.027 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-02-18T01:08:09.340 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up librbd1 (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:09.465 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pecan (1.2.1-2) ... 2024-02-18T01:08:09.588 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/python3-pecan to provide /usr/bin/pecan (pecan) in auto mode 2024-02-18T01:08:09.620 INFO:teuthology.orchestra.run.smithi155.stdout:update-alternatives: using /usr/bin/python3-gunicorn_pecan to provide /usr/bin/gunicorn_pecan (gunicorn_pecan) in auto mode 2024-02-18T01:08:09.640 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-modules-core (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:09.765 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-pastescript (2.0.2-2) ... 2024-02-18T01:08:09.935 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-kubernetes (2.0.0-2ubuntu1) ... 2024-02-18T01:08:10.084 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:10.209 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up rbd-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:10.335 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up python3-rbd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:10.461 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:10.637 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-modules-core (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:10.746 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-pastescript (2.0.2-2) ... 2024-02-18T01:08:11.003 INFO:teuthology.orchestra.run.smithi105.stdout:Adding group ceph....done 2024-02-18T01:08:11.056 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:11.215 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up rbd-fuse (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:11.341 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up python3-rbd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:11.406 INFO:teuthology.orchestra.run.smithi105.stdout:Adding system user ceph....done 2024-02-18T01:08:11.467 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-common (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:11.591 INFO:teuthology.orchestra.run.smithi105.stdout:Setting system user ceph properties....done 2024-02-18T01:08:11.616 INFO:teuthology.orchestra.run.smithi105.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-18T01:08:11.756 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-18T01:08:11.983 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-18T01:08:12.035 INFO:teuthology.orchestra.run.smithi155.stdout:Adding group ceph....done 2024-02-18T01:08:12.280 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up radosgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:12.412 INFO:teuthology.orchestra.run.smithi155.stdout:Adding system user ceph....done 2024-02-18T01:08:12.566 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-18T01:08:12.566 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-18T01:08:12.612 INFO:teuthology.orchestra.run.smithi155.stdout:Setting system user ceph properties....done 2024-02-18T01:08:12.630 INFO:teuthology.orchestra.run.smithi155.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-02-18T01:08:12.769 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-02-18T01:08:12.981 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-test (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:13.008 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-02-18T01:08:13.112 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-base (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:13.319 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up radosgw (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:13.503 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-18T01:08:13.613 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-18T01:08:13.613 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-02-18T01:08:13.793 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:14.003 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-test (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:14.011 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-18T01:08:14.011 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-18T01:08:14.129 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-base (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:14.304 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:14.447 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-osd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:14.495 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-02-18T01:08:14.852 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-18T01:08:14.852 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-18T01:08:14.974 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.123 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.220 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-18T01:08:15.220 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-02-18T01:08:15.266 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mds (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.489 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-18T01:08:15.489 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-18T01:08:15.534 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-diskprediction-local (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.693 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-osd (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.775 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mon (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:15.992 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-18T01:08:15.992 INFO:teuthology.orchestra.run.smithi105.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-18T01:08:16.111 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-18T01:08:16.111 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-02-18T01:08:16.285 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-k8sevents (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:16.411 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-cephadm (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:16.436 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph-mgr-dashboard (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:16.554 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mds (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:16.587 INFO:teuthology.orchestra.run.smithi105.stdout:Setting up ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:16.715 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for systemd (237-3ubuntu10.56) ... 2024-02-18T01:08:16.766 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-18T01:08:16.767 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-02-18T01:08:16.900 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2024-02-18T01:08:17.047 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mon (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:17.252 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-18T01:08:17.252 INFO:teuthology.orchestra.run.smithi155.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-02-18T01:08:17.548 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-k8sevents (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:17.674 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph-mgr-dashboard (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:17.842 INFO:teuthology.orchestra.run.smithi155.stdout:Setting up ceph (16.2.14-548-g81bd20d6-1bionic) ... 2024-02-18T01:08:17.968 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for systemd (237-3ubuntu10.56) ... 2024-02-18T01:08:18.161 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for man-db (2.8.3-2ubuntu0.1) ... 2024-02-18T01:08:18.735 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2024-02-18T01:08:18.852 INFO:teuthology.orchestra.run.smithi105.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.6) ... 2024-02-18T01:08:19.334 INFO:teuthology.orchestra.run.smithi105.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-18T01:08:19.339 DEBUG:teuthology.parallel:result is None 2024-02-18T01:08:20.238 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for ureadahead (0.100.0-21) ... 2024-02-18T01:08:20.347 INFO:teuthology.orchestra.run.smithi155.stdout:Processing triggers for libc-bin (2.27-3ubuntu1.6) ... 2024-02-18T01:08:20.844 INFO:teuthology.orchestra.run.smithi155.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-02-18T01:08:20.849 DEBUG:teuthology.parallel:result is None 2024-02-18T01:08:20.850 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:08:21.005 DEBUG:teuthology.orchestra.run.smithi105:> dpkg-query -W -f '${Version}' ceph 2024-02-18T01:08:21.030 INFO:teuthology.orchestra.run.smithi105.stdout:16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:08:21.031 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:08:21.031 INFO:teuthology.task.install:The correct ceph version 16.2.14-548-g81bd20d6-1bionic is installed. 2024-02-18T01:08:21.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F18.04%2Fx86_64&sha1=81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:08:21.308 DEBUG:teuthology.orchestra.run.smithi155:> dpkg-query -W -f '${Version}' ceph 2024-02-18T01:08:21.334 INFO:teuthology.orchestra.run.smithi155.stdout:16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:08:21.334 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-548-g81bd20d6-1bionic 2024-02-18T01:08:21.334 INFO:teuthology.task.install:The correct ceph version 16.2.14-548-g81bd20d6-1bionic is installed. 2024-02-18T01:08:21.336 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-02-18T01:08:21.336 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:08:21.336 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T01:08:21.351 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:08:21.352 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-02-18T01:08:21.391 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-02-18T01:08:21.391 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:08:21.391 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T01:08:21.408 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T01:08:21.466 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:08:21.466 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/daemon-helper 2024-02-18T01:08:21.481 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-02-18T01:08:21.542 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-02-18T01:08:21.543 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:08:21.543 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T01:08:21.560 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T01:08:21.618 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:08:21.618 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/adjust-ulimits 2024-02-18T01:08:21.633 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-02-18T01:08:21.690 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-02-18T01:08:21.690 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:08:21.690 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T01:08:21.705 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T01:08:21.762 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:08:21.762 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/usr/bin/stdin-killer 2024-02-18T01:08:21.777 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-02-18T01:08:21.834 INFO:teuthology.run_tasks:Running task cephadm... 2024-02-18T01:08:21.911 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms bind msgr2': False, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(MON_DOWN\\)', 'missing hit_sets', 'do not have an application enabled', 'application not enabled on pool', 'pool application', 'mons down', 'out of quorum', 'needs hit_set_type to be set but it is not'], 'mon_bind_msgr2': False, 'sha1': '81bd20d634209c7cb82c18be12b4b5a05643ebf1', 'cephadm_mode': 'cephadm-package'} 2024-02-18T01:08:21.911 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:08:21.912 INFO:tasks.cephadm:Cluster fsid is 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:21.912 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-02-18T01:08:21.912 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '[v1:172.21.15.105:6789]', 'mon.c': '[v1:172.21.15.105:6790]', 'mon.b': '[v1:172.21.15.155:6789]'} 2024-02-18T01:08:21.912 INFO:tasks.cephadm:First mon is mon.a on smithi105 2024-02-18T01:08:21.912 INFO:tasks.cephadm:First mgr is y 2024-02-18T01:08:21.912 INFO:tasks.cephadm:Normalizing hostnames... 2024-02-18T01:08:21.912 DEBUG:teuthology.orchestra.run.smithi105:> sudo hostname $(hostname -s) 2024-02-18T01:08:21.929 DEBUG:teuthology.orchestra.run.smithi155:> sudo hostname $(hostname -s) 2024-02-18T01:08:21.956 DEBUG:teuthology.orchestra.run.smithi105:> sudo mkdir -p /etc/ceph 2024-02-18T01:08:21.986 DEBUG:teuthology.orchestra.run.smithi155:> sudo mkdir -p /etc/ceph 2024-02-18T01:08:22.006 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 777 /etc/ceph 2024-02-18T01:08:22.041 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 777 /etc/ceph 2024-02-18T01:08:22.061 INFO:tasks.cephadm:Writing seed config... 2024-02-18T01:08:22.062 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [global] ms bind msgr2 = False 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [global] ms type = async 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-02-18T01:08:22.063 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-02-18T01:08:22.064 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:08:22.064 DEBUG:teuthology.orchestra.run.smithi105:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-02-18T01:08:22.089 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 358e970a-cdfa-11ee-95ba-87774f69a715 mon election default strategy = 1 ms bind msgr2 = False ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-02-18T01:08:22.090 DEBUG:teuthology.orchestra.run.smithi105:mon.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a.service 2024-02-18T01:08:22.134 DEBUG:teuthology.orchestra.run.smithi105:mgr.y> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y.service 2024-02-18T01:08:22.177 INFO:tasks.cephadm:Bootstrapping... 2024-02-18T01:08:22.177 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 -v bootstrap --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-addrv '[v1:172.21.15.105:6789]' --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:08:22.470 INFO:journalctl@ceph.mon.a.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:08:22.532 INFO:journalctl@ceph.mgr.y.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:08:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:-------------------------------------------------------------------------------- 2024-02-18T01:08:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1', '-v', 'bootstrap', '--fsid', '358e970a-cdfa-11ee-95ba-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-addrv', '[v1:172.21.15.105:6789]', '--skip-admin-label'] 2024-02-18T01:08:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:Verifying podman|docker is present... 2024-02-18T01:08:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:Verifying lvm2 is present... 2024-02-18T01:08:22.541 INFO:teuthology.orchestra.run.smithi105.stderr:Verifying time synchronization is in place... 2024-02-18T01:08:22.548 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T01:08:22.556 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.563 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-18T01:08:22.569 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.578 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: enabled 2024-02-18T01:08:22.587 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.593 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-18T01:08:22.600 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.608 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: enabled 2024-02-18T01:08:22.615 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: active 2024-02-18T01:08:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:Unit ntp.service is enabled and running 2024-02-18T01:08:22.616 INFO:teuthology.orchestra.run.smithi105.stderr:Repeating the final host check... 2024-02-18T01:08:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:docker (/usr/bin/docker) is present 2024-02-18T01:08:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl is present 2024-02-18T01:08:22.617 INFO:teuthology.orchestra.run.smithi105.stderr:lvcreate is present 2024-02-18T01:08:22.624 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2024-02-18T01:08:22.632 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.640 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for chronyd.service: No such file or directory 2024-02-18T01:08:22.649 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.658 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: enabled 2024-02-18T01:08:22.666 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.674 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to get unit file state for ntpd.service: No such file or directory 2024-02-18T01:08:22.681 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: inactive 2024-02-18T01:08:22.690 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: enabled 2024-02-18T01:08:22.699 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: active 2024-02-18T01:08:22.700 INFO:teuthology.orchestra.run.smithi105.stderr:Unit ntp.service is enabled and running 2024-02-18T01:08:22.700 INFO:teuthology.orchestra.run.smithi105.stderr:Host looks OK 2024-02-18T01:08:22.700 INFO:teuthology.orchestra.run.smithi105.stderr:Cluster fsid: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:22.700 INFO:teuthology.orchestra.run.smithi105.stderr:Acquiring lock 140459354021224 on /run/cephadm/358e970a-cdfa-11ee-95ba-87774f69a715.lock 2024-02-18T01:08:22.701 INFO:teuthology.orchestra.run.smithi105.stderr:Lock 140459354021224 acquired on /run/cephadm/358e970a-cdfa-11ee-95ba-87774f69a715.lock 2024-02-18T01:08:22.701 INFO:teuthology.orchestra.run.smithi105.stderr:Verifying IP 172.21.15.105 port 6789 ... 2024-02-18T01:08:22.701 INFO:teuthology.orchestra.run.smithi105.stderr:Base mon IP(s) is [172.21.15.105:6789], mon addrv is [v1:172.21.15.105:6789] 2024-02-18T01:08:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 2024-02-18T01:08:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.105 metric 100 2024-02-18T01:08:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-02-18T01:08:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.105 2024-02-18T01:08:22.706 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.105 metric 100 2024-02-18T01:08:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-02-18T01:08:22.710 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: inet6 ::1/128 scope host 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: inet6 fe80::ec4:7aff:fe8f:d429/64 scope link 2024-02-18T01:08:22.714 INFO:teuthology.orchestra.run.smithi105.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2024-02-18T01:08:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:Mon IP `172.21.15.105` is in CIDR network `172.21.0.0/20` 2024-02-18T01:08:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:Mon IP `172.21.15.105` is in CIDR network `172.21.15.254/32` 2024-02-18T01:08:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.15.254/32'] 2024-02-18T01:08:22.716 INFO:teuthology.orchestra.run.smithi105.stderr:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-02-18T01:08:22.717 INFO:teuthology.orchestra.run.smithi105.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1... 2024-02-18T01:08:23.095 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: 81bd20d634209c7cb82c18be12b4b5a05643ebf1: Pulling from ceph-ci/ceph 2024-02-18T01:08:23.101 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: 0f642e61a11a: Pulling fs layer 2024-02-18T01:08:23.101 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: ec93555fb25f: Pulling fs layer 2024-02-18T01:08:23.816 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: 0f642e61a11a: Download complete 2024-02-18T01:08:25.676 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: ec93555fb25f: Verifying Checksum 2024-02-18T01:08:25.676 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: ec93555fb25f: Download complete 2024-02-18T01:08:26.268 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: 0f642e61a11a: Pull complete 2024-02-18T01:08:36.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: ec93555fb25f: Pull complete 2024-02-18T01:08:36.267 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: Digest: sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9 2024-02-18T01:08:36.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: Status: Downloaded newer image for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:08:36.342 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/docker: quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:08:46.421 INFO:teuthology.orchestra.run.smithi105.stderr:ceph: ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable) 2024-02-18T01:08:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:Ceph version: ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable) 2024-02-18T01:08:46.961 INFO:teuthology.orchestra.run.smithi105.stderr:Extracting ceph user uid/gid from container image... 2024-02-18T01:08:47.832 INFO:teuthology.orchestra.run.smithi105.stderr:stat: 167 167 2024-02-18T01:08:48.365 INFO:teuthology.orchestra.run.smithi105.stderr:Creating initial keys... 2024-02-18T01:08:49.247 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-authtool: AQChWNFlrJCJDhAAobEsLNqSQ7dS8/4PaOVdPg== 2024-02-18T01:08:50.650 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-authtool: AQCiWNFl3WuPJhAAWQLHcnLmS5APP+kRvSLU/A== 2024-02-18T01:08:52.054 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-authtool: AQCkWNFlC87/AhAAmCl0OYOquG14ORBhLoB8sw== 2024-02-18T01:08:52.515 INFO:teuthology.orchestra.run.smithi105.stderr:Creating initial monmap... 2024-02-18T01:08:53.404 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T01:08:53.404 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:53.404 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi105.stderr:monmaptool for a [v1:172.21.15.105:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-02-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/monmaptool: set fsid to 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:53.910 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-02-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:08:53.911 INFO:teuthology.orchestra.run.smithi105.stderr:Creating mon... 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 1 imported monmap: 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: epoch 0 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: last_changed 2024-02-18T01:08:53.400007+0000 2024-02-18T01:08:54.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: created 2024-02-18T01:08:53.400007+0000 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: election_strategy: 1 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 0: v1:172.21.15.105:6789/0 mon.a 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 0 /usr/bin/ceph-mon: set fsid to 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: DB SUMMARY 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-02-18T01:08:54.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.create_if_missing: 1 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.env: 0x55e2da716080 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.fs: Posix File System 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.info_log: 0x55e2dcc29a00 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.statistics: (nil) 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.use_fsync: 0 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T01:08:54.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T01:08:54.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.db_log_dir: 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.write_buffer_manager: 0x55e2dcc2f950 2024-02-18T01:08:54.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.unordered_write: 0 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T01:08:54.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.row_cache: None 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.wal_filter: None 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T01:08:54.853 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_open_files: -1 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T01:08:54.854 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Compression algorithms supported: 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kZSTD supported: 0 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-02-18T01:08:54.855 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.merge_operator: 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_filter: None 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T01:08:54.856 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e2dcb3dd18) 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: index_type: 0 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: index_shortening: 1 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: checksum: 1 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_cache: 0x55e2dcb74d10 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2024-02-18T01:08:54.857 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_cache_options: 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: capacity : 536870912 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_size: 4096 2024-02-18T01:08:54.858 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: partition_filters: 0 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2024-02-18T01:08:54.859 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: verify_compression: 0 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: format_version: 2 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: block_align: 0 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression: NoCompression 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T01:08:54.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T01:08:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.num_levels: 7 2024-02-18T01:08:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T01:08:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T01:08:54.861 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T01:08:54.862 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T01:08:54.862 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T01:08:54.862 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T01:08:54.862 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T01:08:54.863 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T01:08:54.864 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T01:08:54.865 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T01:08:54.866 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.ttl: 2592000 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.804+0000 7f71b2469880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.808+0000 7f71b2469880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.808+0000 7f71b2469880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.812+0000 7f71b2469880 4 rocksdb: DB pointer 0x55e2dcc3d800 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.812+0000 7f719b2dc700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.812+0000 7f719b2dc700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:08:54.867 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.868 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.869 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:08:54.870 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.816+0000 7f71b2469880 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.816+0000 7f71b2469880 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2024-02-18T01:08:54.871 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph-mon: debug 2024-02-18T01:08:54.816+0000 7f71b2469880 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-02-18T01:08:55.328 INFO:teuthology.orchestra.run.smithi105.stderr:create mon.a on 2024-02-18T01:08:55.349 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-02-18T01:08:55.461 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-02-18T01:08:55.583 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target -> /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target. 2024-02-18T01:08:55.583 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target -> /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target. 2024-02-18T01:08:55.849 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to reset failed state of unit ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a.service: Unit ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a.service is not loaded. 2024-02-18T01:08:55.856 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Created symlink /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target.wants/ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715@.service. 2024-02-18T01:08:55.974 INFO:teuthology.orchestra.run.smithi105.stderr:firewalld does not appear to be present 2024-02-18T01:08:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:Not possible to enable service . firewalld.service is not available 2024-02-18T01:08:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mon to start... 2024-02-18T01:08:55.975 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mon... 2024-02-18T01:08:56.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:55 smithi105 systemd[1]: Started Ceph mon.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:08:57.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:57 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.371599+0000 mon.a (mon.0) 0 : [INF] mkfs 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:57.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:57 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.354641+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T01:08:57.511 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcac48a6700 1 Processor -- start 2024-02-18T01:08:57.511 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcac48a6700 1 -- start start 2024-02-18T01:08:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcac48a6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcac0073180 con 0x7fcac0072650 2024-02-18T01:08:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabe59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcac0072650 0x7fcac0072a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60752/0 (socket says 172.21.15.105:60752) 2024-02-18T01:08:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabe59c700 1 -- 172.21.15.105:0/1303134920 learned_addr learned my addr 172.21.15.105:0/1303134920 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:08:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443959962 0 0) 0x7fcac0073180 con 0x7fcac0072650 2024-02-18T01:08:57.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa80033f0 con 0x7fcac0072650 2024-02-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 274780778 0 0) 0x7fcaa80033f0 con 0x7fcac0072650 2024-02-18T01:08:57.513 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0075c30 con 0x7fcac0072650 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcab8001910 con 0x7fcac0072650 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcab8001ba0 con 0x7fcac0072650 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.504+0000 7fcabd59a700 1 -- 172.21.15.105:0/1303134920 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcab8001ef0 con 0x7fcac0072650 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1303134920 >> v1:172.21.15.105:6789/0 conn(0x7fcac0072650 legacy=0x7fcac0072a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1303134920 shutdown_connections 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1303134920 >> 172.21.15.105:0/1303134920 conn(0x7fcac0070f30 msgr2=0x7fcac0071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:08:57.514 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1303134920 shutdown_connections 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1303134920 wait complete. 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 Processor -- start 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- start start 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcac01105f0 con 0x7fcac0072650 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcabe59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcac0072650 0x7fcac010fee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60758/0 (socket says 172.21.15.105:60758) 2024-02-18T01:08:57.515 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcabe59c700 1 -- 172.21.15.105:0/1195218359 learned_addr learned my addr 172.21.15.105:0/1195218359 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:08:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2046998261 0 0) 0x7fcac01105f0 con 0x7fcac0072650 2024-02-18T01:08:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcaa8003c80 con 0x7fcac0072650 2024-02-18T01:08:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1282041478 0 0) 0x7fcaa8003c80 con 0x7fcac0072650 2024-02-18T01:08:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcac01105f0 con 0x7fcac0072650 2024-02-18T01:08:57.516 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcab80018d0 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1699576054 0 0) 0x7fcac01105f0 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac0110780 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcac0110a30 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.508+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcac0114720 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.512+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7fcab8004e30 con 0x7fcac0072650 2024-02-18T01:08:57.518 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.512+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcab8005140 con 0x7fcac0072650 2024-02-18T01:08:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.512+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7fcab8005580 con 0x7fcac0072650 2024-02-18T01:08:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.512+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7fcab8006520 con 0x7fcac0072650 2024-02-18T01:08:57.519 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.512+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcaa0004f50 con 0x7fcac0072650 2024-02-18T01:08:57.520 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.516+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7fcab8006820 con 0x7fcac0072650 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fcaa0005a10 con 0x7fcac0072650 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcaaf7fe700 1 -- 172.21.15.105:0/1195218359 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (unknown 1155462804 0 3911599878) 0x7fcab8006a40 con 0x7fcac0072650 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: cluster: 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: id: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: health: HEALTH_OK 2024-02-18T01:08:57.651 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: services: 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon: 1 daemons, quorum a (age 0.280753s) 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mgr: no daemons active 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: data: 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2024-02-18T01:08:57.653 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: pgs: 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 >> v1:172.21.15.105:6789/0 conn(0x7fcac0072650 legacy=0x7fcac010fee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 shutdown_connections 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 >> 172.21.15.105:0/1195218359 conn(0x7fcac0070f30 msgr2=0x7fcac00fb620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 shutdown_connections 2024-02-18T01:08:57.655 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:57.644+0000 7fcac48a6700 1 -- 172.21.15.105:0/1195218359 wait complete. 2024-02-18T01:08:58.181 INFO:teuthology.orchestra.run.smithi105.stderr:mon is available 2024-02-18T01:08:58.182 INFO:teuthology.orchestra.run.smithi105.stderr:Assimilating anything we can from ceph.conf... 2024-02-18T01:08:58.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.366672+0000 mon.a (mon.0) 2 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T01:08:58.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.368452+0000 mon.a (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.105:6789/0} removed_ranks: {} 2024-02-18T01:08:58.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.376516+0000 mon.a (mon.0) 4 : cluster [DBG] fsmap 2024-02-18T01:08:58.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.380090+0000 mon.a (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-18T01:08:58.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: cluster 2024-02-18T01:08:57.381000+0000 mon.a (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2024-02-18T01:08:58.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:58 smithi105 bash[19087]: audit 2024-02-18T01:08:57.647380+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? v1:172.21.15.105:0/1195218359' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-02-18T01:08:59.290 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.276+0000 7f2a0e29b700 1 Processor -- start 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.276+0000 7f2a0e29b700 1 -- start start 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.276+0000 7f2a0e29b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2a080fc7d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a07fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2a080fbca0 0x7f2a080fc0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60760/0 (socket says 172.21.15.105:60760) 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a07fff700 1 -- 172.21.15.105:0/3855584701 learned_addr learned my addr 172.21.15.105:0/3855584701 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1465842679 0 0) 0x7f2a080fc7d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f00033f0 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1526933355 0 0) 0x7f29f00033f0 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a080fc960 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f29fc001590 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29fc0017b0 con 0x7f2a080fbca0 2024-02-18T01:08:59.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a06ffd700 1 -- 172.21.15.105:0/3855584701 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f29fc001b00 con 0x7f2a080fbca0 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a0e29b700 1 -- 172.21.15.105:0/3855584701 >> v1:172.21.15.105:6789/0 conn(0x7f2a080fbca0 legacy=0x7f2a080fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a0e29b700 1 -- 172.21.15.105:0/3855584701 shutdown_connections 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a0e29b700 1 -- 172.21.15.105:0/3855584701 >> 172.21.15.105:0/3855584701 conn(0x7f2a080f5550 msgr2=0x7f2a080f7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a0e29b700 1 -- 172.21.15.105:0/3855584701 shutdown_connections 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.280+0000 7f2a0e29b700 1 -- 172.21.15.105:0/3855584701 wait complete. 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a0e29b700 1 Processor -- start 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a0e29b700 1 -- start start 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a0e29b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2a0810c1d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a07fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f2a080fbca0 0x7f2a0810bac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60764/0 (socket says 172.21.15.105:60764) 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a07fff700 1 -- 172.21.15.105:0/342868738 learned_addr learned my addr 172.21.15.105:0/342868738 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 959033036 0 0) 0x7f2a0810c1d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29f0003c80 con 0x7f2a080fbca0 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3050667832 0 0) 0x7f29f0003c80 con 0x7f2a080fbca0 2024-02-18T01:08:59.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2a0810c1d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f29fc001490 con 0x7f2a080fbca0 2024-02-18T01:08:59.293 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3162890832 0 0) 0x7f2a0810c1d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0810c360 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0810c610 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(0 keys) v1 ==== 4+0+0 (unknown 0 0 0) 0x7f29fc004a20 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f29fc004d30 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a08110300 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f29fc005170 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.284+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f29fc005780 con 0x7f2a080fbca0 2024-02-18T01:08:59.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.288+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a08046ce0 con 0x7f2a080fbca0 2024-02-18T01:08:59.296 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.292+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f29fc005940 con 0x7f2a080fbca0 2024-02-18T01:08:59.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.420+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2a081020f0 con 0x7f2a080fbca0 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.424+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 10 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f29fc0062d0 con 0x7f2a080fbca0 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.424+0000 7f2a05775700 1 -- 172.21.15.105:0/342868738 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+343 (unknown 1213389831 0 1339778329) 0x7f2a081020f0 con 0x7f2a080fbca0 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [global] 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: fsid = 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_host = [v1:172.21.15.105:6789] 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-18T01:08:59.434 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [mgr] 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [osd] 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.428+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 >> v1:172.21.15.105:6789/0 conn(0x7f2a080fbca0 legacy=0x7f2a0810bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.428+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 shutdown_connections 2024-02-18T01:08:59.435 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.428+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 >> 172.21.15.105:0/342868738 conn(0x7f2a080f5550 msgr2=0x7f2a080f6150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:08:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.428+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 shutdown_connections 2024-02-18T01:08:59.436 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:08:59.428+0000 7f2a0e29b700 1 -- 172.21.15.105:0/342868738 wait complete. 2024-02-18T01:08:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:59 smithi105 bash[19087]: audit 2024-02-18T01:08:59.424699+0000 mon.a (mon.0) 8 : audit [INF] from='client.? v1:172.21.15.105:0/342868738' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T01:08:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:08:59 smithi105 bash[19087]: audit 2024-02-18T01:08:59.428557+0000 mon.a (mon.0) 9 : audit [INF] from='client.? v1:172.21.15.105:0/342868738' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-02-18T01:09:00.005 INFO:teuthology.orchestra.run.smithi105.stderr:Generating new minimal ceph.conf... 2024-02-18T01:09:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.984+0000 7f4413cd6700 1 Processor -- start 2024-02-18T01:09:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.984+0000 7f4413cd6700 1 -- start start 2024-02-18T01:09:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f440c0fe3c0 con 0x7f440c0fd890 2024-02-18T01:09:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4411a72700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f440c0fd890 0x7f440c0fdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48610/0 (socket says 172.21.15.105:48610) 2024-02-18T01:09:00.994 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4411a72700 1 -- 172.21.15.105:0/2333463175 learned_addr learned my addr 172.21.15.105:0/2333463175 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401991279 0 0) 0x7f440c0fe3c0 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f40033f0 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 779127606 0 0) 0x7f43f40033f0 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c102e10 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4404001910 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f4404001fb0 con 0x7f440c0fd890 2024-02-18T01:09:00.995 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4410a70700 1 -- 172.21.15.105:0/2333463175 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f44040059b0 con 0x7f440c0fd890 2024-02-18T01:09:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- 172.21.15.105:0/2333463175 >> v1:172.21.15.105:6789/0 conn(0x7f440c0fd890 legacy=0x7f440c0fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- 172.21.15.105:0/2333463175 shutdown_connections 2024-02-18T01:09:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- 172.21.15.105:0/2333463175 >> 172.21.15.105:0/2333463175 conn(0x7f440c070f30 msgr2=0x7f440c071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- 172.21.15.105:0/2333463175 shutdown_connections 2024-02-18T01:09:00.997 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- 172.21.15.105:0/2333463175 wait complete. 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 Processor -- start 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.988+0000 7f4413cd6700 1 -- start start 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4413cd6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f440c10c970 con 0x7f440c0fd890 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4411a72700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f440c0fd890 0x7f440c10c260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48626/0 (socket says 172.21.15.105:48626) 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4411a72700 1 -- 172.21.15.105:0/2381250866 learned_addr learned my addr 172.21.15.105:0/2381250866 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017998722 0 0) 0x7f440c10c970 con 0x7f440c0fd890 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43f4005260 con 0x7f440c0fd890 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2059638580 0 0) 0x7f43f4005260 con 0x7f440c0fd890 2024-02-18T01:09:00.998 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f440c10c970 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4404001250 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1551327538 0 0) 0x7f440c10c970 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c10cb00 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c10cdb0 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c110aa0 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f4404004e80 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4404006490 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f4404006950 con 0x7f440c0fd890 2024-02-18T01:09:00.999 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f4404007870 con 0x7f440c0fd890 2024-02-18T01:09:01.000 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.992+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f8004f50 con 0x7f440c0fd890 2024-02-18T01:09:01.002 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:00.996+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f4404005c90 con 0x7f440c0fd890 2024-02-18T01:09:01.131 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f43f8005200 con 0x7f440c0fd890 2024-02-18T01:09:01.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f440affd700 1 -- 172.21.15.105:0/2381250866 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+148 (unknown 1452402520 0 1685388076) 0x7f4404005e30 con 0x7f440c0fd890 2024-02-18T01:09:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 >> v1:172.21.15.105:6789/0 conn(0x7f440c0fd890 legacy=0x7f440c10c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:01.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 shutdown_connections 2024-02-18T01:09:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 >> 172.21.15.105:0/2381250866 conn(0x7f440c070f30 msgr2=0x7f440c0facf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.124+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 shutdown_connections 2024-02-18T01:09:01.134 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:01.128+0000 7f4413cd6700 1 -- 172.21.15.105:0/2381250866 wait complete. 2024-02-18T01:09:01.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:01 smithi105 bash[19087]: audit 2024-02-18T01:09:01.127637+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.105:0/2381250866' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:09:01.665 INFO:teuthology.orchestra.run.smithi105.stderr:Restarting the monitor... 2024-02-18T01:09:01.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:01 smithi105 systemd[1]: Stopping Ceph mon.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:09:01.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:01 smithi105 bash[19613]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mon.a 2024-02-18T01:09:01.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:01 smithi105 bash[19087]: debug 2024-02-18T01:09:01.840+0000 7fc03b0ff700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T01:09:01.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:01 smithi105 bash[19087]: debug 2024-02-18T01:09:01.840+0000 7fc03b0ff700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-02-18T01:09:02.599 INFO:teuthology.orchestra.run.smithi105.stderr:Setting public_network to 172.21.15.254/32,172.21.0.0/20 in mon config section 2024-02-18T01:09:02.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:02 smithi105 bash[19613]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mon-a 2024-02-18T01:09:02.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:02 smithi105 systemd[1]: Stopped Ceph mon.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:09:02.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:02 smithi105 systemd[1]: Started Ceph mon.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.899+0000 7fbf7666c880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.899+0000 7fbf7666c880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 8 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.899+0000 7fbf7666c880 0 pidfile_write: ignore empty --pid-file 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 0 load: jerasure load: lrc load: isa 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T01:09:03.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: DB SUMMARY 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000004.sst 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000006.log size: 85977 ; 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.env: 0x556ce5d82080 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.fs: Posix File System 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.info_log: 0x556ce808d9c0 2024-02-18T01:09:03.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.statistics: (nil) 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.use_fsync: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T01:09:03.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.db_log_dir: 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.write_buffer_manager: 0x556ce8d8c360 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T01:09:03.996 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.unordered_write: 0 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.row_cache: None 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.wal_filter: None 2024-02-18T01:09:03.997 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T01:09:03.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T01:09:03.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T01:09:03.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T01:09:03.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_open_files: -1 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T01:09:04.000 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Compression algorithms supported: 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kZSTD supported: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T01:09:04.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.merge_operator: 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_filter: None 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556ce803bd10) 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cache_index_and_filter_blocks: 1 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: pin_top_level_index_and_filter: 1 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: index_type: 0 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: data_block_index_type: 0 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: index_shortening: 1 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T01:09:04.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: hash_index_allow_collision: 1 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: checksum: 1 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: no_block_cache: 0 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_cache: 0x556ce8072f10 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_cache_name: BinnedLRUCache 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_cache_options: 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: capacity : 536870912 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: num_shard_bits : 4 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: strict_capacity_limit : 0 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: high_pri_pool_ratio: 0.000 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_cache_compressed: (nil) 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: persistent_cache: (nil) 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_size: 4096 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_size_deviation: 10 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_restart_interval: 16 2024-02-18T01:09:04.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: index_block_restart_interval: 1 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: metadata_block_size: 4096 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: partition_filters: 0 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: use_delta_encoding: 1 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: whole_key_filtering: 1 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: verify_compression: 0 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: read_amp_bytes_per_bit: 0 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: format_version: 2 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: enable_index_compression: 1 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: block_align: 0 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T01:09:04.004 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression: NoCompression 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.num_levels: 7 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T01:09:04.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T01:09:04.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T01:09:04.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T01:09:04.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.ttl: 2592000 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.903+0000 7fbf7666c880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218543911447, "job": 1, "event": "recovery_started", "log_files": [6]} 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T01:09:04.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218543912674, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 82402, "table_properties": {"data_size": 80676, "index_size": 331, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 581, "raw_key_size": 10041, "raw_average_key_size": 47, "raw_value_size": 74979, "raw_average_value_size": 357, "num_data_blocks": 11, "num_entries": 210, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708218543, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.907+0000 7fbf7666c880 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218543913768, "job": 1, "event": "recovery_finished"} 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 4 rocksdb: DB pointer 0x556ce813b800 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf5d45e700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf5d45e700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** DB Stats ** 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:09:04.011 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** Compaction Stats [default] ** 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: L0 2/0 82.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Sum 2/0 82.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** Compaction Stats [default] ** 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:09:04.012 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Cumulative compaction: 0.00 GB write, 13.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Interval compaction: 0.00 GB write, 13.13 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** Compaction Stats [default] ** 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: L0 2/0 82.09 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Sum 2/0 82.09 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:09:04.013 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** Compaction Stats [default] ** 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 85.9 0.00 0.00 1 0.001 0 0 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Cumulative compaction: 0.00 GB write, 12.99 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 0 starting mon.a rank 0 at public addrs v1:172.21.15.105:6789/0 at bind addrs v1:172.21.15.105:6789/0 mon_data /var/lib/ceph/mon/ceph-a fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:09:04.014 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 1 mon.a@-1(???) e1 preinit fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 0 mon.a@-1(???).mds e1 new map 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 0 mon.a@-1(???).mds e1 print_map 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: e1 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: legacy client fscid: -1 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: No filesystems configured 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.911+0000 7fbf7666c880 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: audit 2024-02-18T01:09:01.127637+0000 mon.a (mon.0) 10 : audit [DBG] from='client.? v1:172.21.15.105:0/2381250866' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: debug 2024-02-18T01:09:03.915+0000 7fbf7666c880 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cluster 2024-02-18T01:09:03.921091+0000 mon.a (mon.0) 1 : cluster [INF] mon.a is new leader, mons a in quorum (ranks 0) 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cluster 2024-02-18T01:09:03.921161+0000 mon.a (mon.0) 2 : cluster [DBG] monmap e1: 1 mons at {a=v1:172.21.15.105:6789/0} removed_ranks: {} 2024-02-18T01:09:04.015 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cluster 2024-02-18T01:09:03.921635+0000 mon.a (mon.0) 3 : cluster [DBG] fsmap 2024-02-18T01:09:04.016 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cluster 2024-02-18T01:09:03.921677+0000 mon.a (mon.0) 4 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2024-02-18T01:09:04.016 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:03 smithi105 bash[19734]: cluster 2024-02-18T01:09:03.922203+0000 mon.a (mon.0) 5 : cluster [DBG] mgrmap e1: no daemons active 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee389e3700 1 Processor -- start 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee389e3700 1 -- start start 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee389e3700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee340fc120 con 0x7fee3406fbf0 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3259c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fee3406fbf0 0x7fee34100730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48630/0 (socket says 172.21.15.105:48630) 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3259c700 1 -- 172.21.15.105:0/712106985 learned_addr learned my addr 172.21.15.105:0/712106985 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1215708058 0 0) 0x7fee340fc120 con 0x7fee3406fbf0 2024-02-18T01:09:04.132 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee140033f0 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2714763808 0 0) 0x7fee140033f0 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.123+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee34102e60 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee2c001910 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fee2c001fb0 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee3159a700 1 -- 172.21.15.105:0/712106985 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee2c0059b0 con 0x7fee3406fbf0 2024-02-18T01:09:04.133 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- 172.21.15.105:0/712106985 >> v1:172.21.15.105:6789/0 conn(0x7fee3406fbf0 legacy=0x7fee34100730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- 172.21.15.105:0/712106985 shutdown_connections 2024-02-18T01:09:04.134 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- 172.21.15.105:0/712106985 >> 172.21.15.105:0/712106985 conn(0x7fee340f5db0 msgr2=0x7fee340f81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- 172.21.15.105:0/712106985 shutdown_connections 2024-02-18T01:09:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- 172.21.15.105:0/712106985 wait complete. 2024-02-18T01:09:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 Processor -- start 2024-02-18T01:09:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- start start 2024-02-18T01:09:04.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee389e3700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee3410c990 con 0x7fee3406fbf0 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee3259c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fee3406fbf0 0x7fee3410c280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48638/0 (socket says 172.21.15.105:48638) 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee3259c700 1 -- 172.21.15.105:0/2276831066 learned_addr learned my addr 172.21.15.105:0/2276831066 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133041485 0 0) 0x7fee3410c990 con 0x7fee3406fbf0 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee14005260 con 0x7fee3406fbf0 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3332444380 0 0) 0x7fee14005260 con 0x7fee3406fbf0 2024-02-18T01:09:04.136 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee3410c990 con 0x7fee3406fbf0 2024-02-18T01:09:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.127+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee2c001250 con 0x7fee3406fbf0 2024-02-18T01:09:04.137 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2916883646 0 0) 0x7fee3410c990 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3410cb20 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee3410cdd0 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee34110ac0 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fee2c004da0 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee2c0064d0 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7fee2c006990 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7fee2c007930 con 0x7fee3406fbf0 2024-02-18T01:09:04.138 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.131+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee18004f50 con 0x7fee3406fbf0 2024-02-18T01:09:04.140 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.135+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7fee2c005ca0 con 0x7fee3406fbf0 2024-02-18T01:09:04.271 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.263+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fee18005200 con 0x7fee3406fbf0 2024-02-18T01:09:04.276 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee2b7fe700 1 -- 172.21.15.105:0/2276831066 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 141+0+0 (unknown 223993786 0 0) 0x7fee2c006e20 con 0x7fee3406fbf0 2024-02-18T01:09:04.277 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 >> v1:172.21.15.105:6789/0 conn(0x7fee3406fbf0 legacy=0x7fee3410c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 shutdown_connections 2024-02-18T01:09:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 >> 172.21.15.105:0/2276831066 conn(0x7fee340f5db0 msgr2=0x7fee340f69b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:04.278 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 shutdown_connections 2024-02-18T01:09:04.279 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:04.271+0000 7fee389e3700 1 -- 172.21.15.105:0/2276831066 wait complete. 2024-02-18T01:09:04.893 INFO:teuthology.orchestra.run.smithi105.stderr:Wrote config to /etc/ceph/ceph.conf 2024-02-18T01:09:04.894 INFO:teuthology.orchestra.run.smithi105.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:09:04.894 INFO:teuthology.orchestra.run.smithi105.stderr:Creating mgr... 2024-02-18T01:09:04.894 INFO:teuthology.orchestra.run.smithi105.stderr:Verifying port 9283 ... 2024-02-18T01:09:05.048 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Failed to reset failed state of unit ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y.service: Unit ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y.service is not loaded. 2024-02-18T01:09:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:systemctl: Created symlink /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715.target.wants/ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-358e970a-cdfa-11ee-95ba-87774f69a715@.service. 2024-02-18T01:09:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:firewalld does not appear to be present 2024-02-18T01:09:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:Not possible to enable service . firewalld.service is not available 2024-02-18T01:09:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:firewalld does not appear to be present 2024-02-18T01:09:05.185 INFO:teuthology.orchestra.run.smithi105.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2024-02-18T01:09:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mgr to start... 2024-02-18T01:09:05.186 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mgr... 2024-02-18T01:09:05.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:05 smithi105 bash[19734]: audit 2024-02-18T01:09:04.272981+0000 mon.a (mon.0) 6 : audit [INF] from='client.? v1:172.21.15.105:0/2276831066' entity='client.admin' 2024-02-18T01:09:05.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:05 smithi105 systemd[1]: Started Ceph mgr.y for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:09:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.867+0000 7f3ff313a700 1 Processor -- start 2024-02-18T01:09:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.867+0000 7f3ff313a700 1 -- start start 2024-02-18T01:09:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3fec0fa8e0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff0ed6700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3fec0f9db0 0x7f3fec0fa1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48664/0 (socket says 172.21.15.105:48664) 2024-02-18T01:09:06.878 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff0ed6700 1 -- 172.21.15.105:0/894386740 learned_addr learned my addr 172.21.15.105:0/894386740 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 355664370 0 0) 0x7f3fe8001490 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd40033f0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142786922 0 0) 0x7f3fd40033f0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec0ff1f0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3fe8001aa0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f3fe8004af0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.879 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3fe37fe700 1 -- 172.21.15.105:0/894386740 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3fe8005b50 con 0x7f3fec0f9db0 2024-02-18T01:09:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- 172.21.15.105:0/894386740 >> v1:172.21.15.105:6789/0 conn(0x7f3fec0f9db0 legacy=0x7f3fec0fa1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- 172.21.15.105:0/894386740 shutdown_connections 2024-02-18T01:09:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- 172.21.15.105:0/894386740 >> 172.21.15.105:0/894386740 conn(0x7f3fec0f5550 msgr2=0x7f3fec0f7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- 172.21.15.105:0/894386740 shutdown_connections 2024-02-18T01:09:06.881 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- 172.21.15.105:0/894386740 wait complete. 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 Processor -- start 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- start start 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff313a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3fec10a880 con 0x7f3fec0f9db0 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff0ed6700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3fec0f9db0 0x7f3fec10a170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48676/0 (socket says 172.21.15.105:48676) 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.871+0000 7f3ff0ed6700 1 -- 172.21.15.105:0/2815422971 learned_addr learned my addr 172.21.15.105:0/2815422971 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044566313 0 0) 0x7f3fec10a880 con 0x7f3fec0f9db0 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fd4005260 con 0x7f3fec0f9db0 2024-02-18T01:09:06.882 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3858435800 0 0) 0x7f3fd4005260 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3fec10a880 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3fe8004c60 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1296448340 0 0) 0x7f3fec10a880 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fec10aa10 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fec10acc0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fec10e9b0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f3fe8005020 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3fe80064b0 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f3fe8006970 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f3fe8007920 con 0x7f3fec0f9db0 2024-02-18T01:09:06.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.875+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd8004f50 con 0x7f3fec0f9db0 2024-02-18T01:09:06.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:06.879+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f3fec0fa8e0 con 0x7f3fec0f9db0 2024-02-18T01:09:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.027+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3fd8005200 con 0x7f3fec0f9db0 2024-02-18T01:09:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.027+0000 7f3fe1ffb700 1 -- 172.21.15.105:0/2815422971 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 523397627) 0x7f3fe8005e40 con 0x7f3fec0f9db0 2024-02-18T01:09:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:07.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsid": "358e970a-cdfa-11ee-95ba-87774f69a715", 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "health": { 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 0 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "a" 2024-02-18T01:09:07.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_age": 3, 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T01:09:07.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T01:09:07.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T01:09:07.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T01:09:07.046 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T01:09:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T01:09:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T01:09:07.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "available": false, 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "iostat", 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "nfs", 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "restful" 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T01:09:07.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modified": "2024-02-18T01:08:57.372205+0000", 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.031+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 >> v1:172.21.15.105:6789/0 conn(0x7f3fec0f9db0 legacy=0x7f3fec10a170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.031+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 shutdown_connections 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.031+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 >> 172.21.15.105:0/2815422971 conn(0x7f3fec0f5550 msgr2=0x7f3fec101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.031+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 shutdown_connections 2024-02-18T01:09:07.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:07.031+0000 7f3ff313a700 1 -- 172.21.15.105:0/2815422971 wait complete. 2024-02-18T01:09:07.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:07 smithi105 bash[19734]: audit 2024-02-18T01:09:07.032512+0000 mon.a (mon.0) 7 : audit [DBG] from='client.? v1:172.21.15.105:0/2815422971' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T01:09:07.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:06 smithi105 bash[20046]: debug 2024-02-18T01:09:06.935+0000 7fe26f037680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:09:07.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:07 smithi105 bash[20046]: debug 2024-02-18T01:09:07.035+0000 7fe26f037680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:09:07.667 INFO:teuthology.orchestra.run.smithi105.stderr:mgr not available, waiting (1/15)... 2024-02-18T01:09:07.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:07 smithi105 bash[20046]: debug 2024-02-18T01:09:07.479+0000 7fe26f037680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:09:08.698 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:08 smithi105 bash[20046]: debug 2024-02-18T01:09:08.367+0000 7fe26f037680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:09:08.698 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:08 smithi105 bash[20046]: debug 2024-02-18T01:09:08.455+0000 7fe26f037680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:09:08.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:08 smithi105 bash[20046]: debug 2024-02-18T01:09:08.691+0000 7fe26f037680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:09:10.031 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:09 smithi105 bash[20046]: debug 2024-02-18T01:09:09.771+0000 7fe26f037680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:09:10.031 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:09 smithi105 bash[20046]: debug 2024-02-18T01:09:09.927+0000 7fe26f037680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:10.353 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:10 smithi105 bash[20046]: debug 2024-02-18T01:09:10.023+0000 7fe26f037680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:10.353 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:10 smithi105 bash[20046]: debug 2024-02-18T01:09:10.131+0000 7fe26f037680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:09:10.676 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:10 smithi105 bash[20046]: debug 2024-02-18T01:09:10.347+0000 7fe26f037680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:09:10.888 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.879+0000 7f92d2145700 1 Processor -- start 2024-02-18T01:09:10.889 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.879+0000 7f92d2145700 1 -- start start 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f92cc0fd7e0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92cb7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92cc0fccb0 0x7f92cc0fd0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57006/0 (socket says 172.21.15.105:57006) 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92cb7fe700 1 -- 172.21.15.105:0/741709206 learned_addr learned my addr 172.21.15.105:0/741709206 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 58501509 0 0) 0x7f92cc0fd7e0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92b40033f0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.890 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1263052327 0 0) 0x7f92b40033f0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc0fd970 con 0x7f92cc0fccb0 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f92bc001590 con 0x7f92cc0fccb0 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f92bc001d00 con 0x7f92cc0fccb0 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92ca7fc700 1 -- 172.21.15.105:0/741709206 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f92bc009b10 con 0x7f92cc0fccb0 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- 172.21.15.105:0/741709206 >> v1:172.21.15.105:6789/0 conn(0x7f92cc0fccb0 legacy=0x7f92cc0fd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:10.891 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- 172.21.15.105:0/741709206 shutdown_connections 2024-02-18T01:09:10.892 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- 172.21.15.105:0/741709206 >> 172.21.15.105:0/741709206 conn(0x7f92cc0f5550 msgr2=0x7f92cc0f7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:10.892 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- 172.21.15.105:0/741709206 shutdown_connections 2024-02-18T01:09:10.892 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.883+0000 7f92d2145700 1 -- 172.21.15.105:0/741709206 wait complete. 2024-02-18T01:09:10.892 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 Processor -- start 2024-02-18T01:09:10.893 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 -- start start 2024-02-18T01:09:10.893 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f92cc110ce0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.893 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92cb7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92cc0fccb0 0x7f92cc1105d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57018/0 (socket says 172.21.15.105:57018) 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92cb7fe700 1 -- 172.21.15.105:0/88090520 learned_addr learned my addr 172.21.15.105:0/88090520 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1097468980 0 0) 0x7f92cc110ce0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92b4005210 con 0x7f92cc0fccb0 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3008933129 0 0) 0x7f92b4005210 con 0x7f92cc0fccb0 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f92cc110ce0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f92bc000fe0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2755677524 0 0) 0x7f92cc110ce0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc110e70 con 0x7f92cc0fccb0 2024-02-18T01:09:10.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc111120 con 0x7f92cc0fccb0 2024-02-18T01:09:10.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc114e10 con 0x7f92cc0fccb0 2024-02-18T01:09:10.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc046ce0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f92bc001430 con 0x7f92cc0fccb0 2024-02-18T01:09:10.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f92bc004fa0 con 0x7f92cc0fccb0 2024-02-18T01:09:10.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 1) v1 ==== 495+0+0 (unknown 2835230805 0 0) 0x7f92bc005460 con 0x7f92cc0fccb0 2024-02-18T01:09:10.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.887+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f92bc006440 con 0x7f92cc0fccb0 2024-02-18T01:09:10.900 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:10.891+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73271 (unknown 1092875540 0 259193764) 0x7f92bc004690 con 0x7f92cc0fccb0 2024-02-18T01:09:10.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:10 smithi105 bash[20046]: debug 2024-02-18T01:09:10.707+0000 7fe26f037680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:09:10.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:10 smithi105 bash[20046]: debug 2024-02-18T01:09:10.847+0000 7fe26f037680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:11.036 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.027+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f92cc1008a0 con 0x7f92cc0fccb0 2024-02-18T01:09:11.036 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.027+0000 7f92c8ff9700 1 -- 172.21.15.105:0/88090520 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 801263350) 0x7f92bc0015a0 con 0x7f92cc0fccb0 2024-02-18T01:09:11.037 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.037 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:11.037 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsid": "358e970a-cdfa-11ee-95ba-87774f69a715", 2024-02-18T01:09:11.037 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "health": { 2024-02-18T01:09:11.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T01:09:11.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T01:09:11.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T01:09:11.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.038 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T01:09:11.039 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 0 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "a" 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_age": 7, 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:11.040 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T01:09:11.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T01:09:11.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T01:09:11.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:11.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T01:09:11.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T01:09:11.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T01:09:11.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T01:09:11.043 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T01:09:11.046 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T01:09:11.046 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T01:09:11.047 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "available": false, 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "iostat", 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "nfs", 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "restful" 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modified": "2024-02-18T01:08:57.372205+0000", 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:11.048 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:11.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T01:09:11.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:11.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.031+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 >> v1:172.21.15.105:6789/0 conn(0x7f92cc0fccb0 legacy=0x7f92cc1105d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:11.050 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.031+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 shutdown_connections 2024-02-18T01:09:11.050 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.031+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 >> 172.21.15.105:0/88090520 conn(0x7f92cc0f5550 msgr2=0x7f92cc0f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:11.050 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.031+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 shutdown_connections 2024-02-18T01:09:11.050 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:11.031+0000 7f92d2145700 1 -- 172.21.15.105:0/88090520 wait complete. 2024-02-18T01:09:11.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:11 smithi105 bash[19734]: audit 2024-02-18T01:09:11.032831+0000 mon.a (mon.0) 8 : audit [DBG] from='client.? v1:172.21.15.105:0/88090520' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T01:09:11.631 INFO:teuthology.orchestra.run.smithi105.stderr:mgr not available, waiting (2/15)... 2024-02-18T01:09:11.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:11 smithi105 bash[20046]: debug 2024-02-18T01:09:11.723+0000 7fe26f037680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:09:11.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:11 smithi105 bash[20046]: debug 2024-02-18T01:09:11.823+0000 7fe26f037680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:09:11.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:11 smithi105 bash[20046]: debug 2024-02-18T01:09:11.935+0000 7fe26f037680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:09:12.432 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:12 smithi105 bash[20046]: debug 2024-02-18T01:09:12.143+0000 7fe26f037680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:09:12.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:12 smithi105 bash[20046]: debug 2024-02-18T01:09:12.427+0000 7fe26f037680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:09:12.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:12 smithi105 bash[20046]: debug 2024-02-18T01:09:12.579+0000 7fe26f037680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:09:13.066 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:12 smithi105 bash[20046]: debug 2024-02-18T01:09:12.747+0000 7fe26f037680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:13.066 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:12 smithi105 bash[20046]: debug 2024-02-18T01:09:12.955+0000 7fe26f037680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:09:13.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[20046]: debug 2024-02-18T01:09:13.059+0000 7fe26f037680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: cluster 2024-02-18T01:09:13.083059+0000 mon.a (mon.0) 9 : cluster [INF] Activating manager daemon y 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: cluster 2024-02-18T01:09:13.089273+0000 mon.a (mon.0) 10 : cluster [DBG] mgrmap e2: y(active, starting, since 0.00633162s) 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.090764+0000 mon.a (mon.0) 11 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091043+0000 mon.a (mon.0) 12 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091222+0000 mon.a (mon.0) 13 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091415+0000 mon.a (mon.0) 14 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091597+0000 mon.a (mon.0) 15 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091799+0000 mon.a (mon.0) 16 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.091967+0000 mon.a (mon.0) 17 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.092598+0000 mon.a (mon.0) 18 : audit [DBG] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: cluster 2024-02-18T01:09:13.097589+0000 mon.a (mon.0) 19 : cluster [INF] Manager daemon y is now available 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.120039+0000 mon.a (mon.0) 20 : audit [INF] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' 2024-02-18T01:09:13.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.126544+0000 mon.a (mon.0) 21 : audit [INF] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:09:13.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:13 smithi105 bash[19734]: audit 2024-02-18T01:09:13.131236+0000 mon.a (mon.0) 22 : audit [INF] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:09:14.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:14 smithi105 bash[19734]: audit 2024-02-18T01:09:13.139080+0000 mon.a (mon.0) 23 : audit [INF] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' 2024-02-18T01:09:14.450 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:14 smithi105 bash[19734]: audit 2024-02-18T01:09:13.147012+0000 mon.a (mon.0) 24 : audit [INF] from='mgr.14100 v1:172.21.15.105:0/1572286157' entity='mgr.y' 2024-02-18T01:09:14.451 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:14 smithi105 bash[19734]: cluster 2024-02-18T01:09:14.094370+0000 mon.a (mon.0) 25 : cluster [DBG] mgrmap e3: y(active, since 1.01143s) 2024-02-18T01:09:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.747+0000 7f4cce32f700 1 Processor -- start 2024-02-18T01:09:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.747+0000 7f4cce32f700 1 -- start start 2024-02-18T01:09:14.757 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.747+0000 7f4cce32f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cc80fc1c0 con 0x7f4cc80fb690 2024-02-18T01:09:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc7fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4cc80fb690 0x7f4cc80fbab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57066/0 (socket says 172.21.15.105:57066) 2024-02-18T01:09:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc7fff700 1 -- 172.21.15.105:0/4014307677 learned_addr learned my addr 172.21.15.105:0/4014307677 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:14.758 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285564237 0 0) 0x7f4cc80fc1c0 con 0x7f4cc80fb690 2024-02-18T01:09:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca80033f0 con 0x7f4cc80fb690 2024-02-18T01:09:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1510249602 0 0) 0x7f4ca80033f0 con 0x7f4cc80fb690 2024-02-18T01:09:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc8100c10 con 0x7f4cc80fb690 2024-02-18T01:09:14.759 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4cb8001590 con 0x7f4cc80fb690 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f4cb8001d00 con 0x7f4cc80fb690 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cc6ffd700 1 -- 172.21.15.105:0/4014307677 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4cb8005700 con 0x7f4cc80fb690 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- 172.21.15.105:0/4014307677 >> v1:172.21.15.105:6789/0 conn(0x7f4cc80fb690 legacy=0x7f4cc80fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- 172.21.15.105:0/4014307677 shutdown_connections 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- 172.21.15.105:0/4014307677 >> 172.21.15.105:0/4014307677 conn(0x7f4cc80f6e90 msgr2=0x7f4cc80f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- 172.21.15.105:0/4014307677 shutdown_connections 2024-02-18T01:09:14.760 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- 172.21.15.105:0/4014307677 wait complete. 2024-02-18T01:09:14.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 Processor -- start 2024-02-18T01:09:14.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- start start 2024-02-18T01:09:14.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.751+0000 7f4cce32f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4cc810ac10 con 0x7f4cc810a0e0 2024-02-18T01:09:14.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc7fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4cc810a0e0 0x7f4cc810a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57082/0 (socket says 172.21.15.105:57082) 2024-02-18T01:09:14.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc7fff700 1 -- 172.21.15.105:0/3012413636 learned_addr learned my addr 172.21.15.105:0/3012413636 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647778829 0 0) 0x7f4cc810ac10 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ca8005210 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3322601781 0 0) 0x7f4ca8005210 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4cc810ac10 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4cb8004b90 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1330110766 0 0) 0x7f4cc810ac10 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc810f6d0 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cc810e6c0 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cc810ebb0 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f4cb8000f90 con 0x7f4cc810a0e0 2024-02-18T01:09:14.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4cb80060a0 con 0x7f4cc810a0e0 2024-02-18T01:09:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 3) v1 ==== 38465+0+0 (unknown 3662677962 0 0) 0x7f4cb800f850 con 0x7f4cc810a0e0 2024-02-18T01:09:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f4cb803f790 con 0x7f4cc810a0e0 2024-02-18T01:09:14.764 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.755+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cac004f50 con 0x7f4cc810a0e0 2024-02-18T01:09:14.772 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:14.763+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4cb8010f30 con 0x7f4cc810a0e0 2024-02-18T01:09:15.148 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.139+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mgrmap(e 4) v1 ==== 38571+0+0 (unknown 3881454507 0 0) 0x7f4cb80059f0 con 0x7f4cc810a0e0 2024-02-18T01:09:15.221 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f4cac005200 con 0x7f4cc810a0e0 2024-02-18T01:09:15.223 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cc57fa700 1 -- 172.21.15.105:0/3012413636 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1241 (unknown 4201413639 0 1135522087) 0x7f4cb803eae0 con 0x7f4cc810a0e0 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsid": "358e970a-cdfa-11ee-95ba-87774f69a715", 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "health": { 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "checks": {}, 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mutes": [] 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "election_epoch": 5, 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum": [ 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 0 2024-02-18T01:09:15.224 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:15.225 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_names": [ 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "a" 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "quorum_age": 11, 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "monmap": { 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2024-02-18T01:09:15.226 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_mons": 1 2024-02-18T01:09:15.227 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.227 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osdmap": { 2024-02-18T01:09:15.227 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_osds": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_up_osds": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_up_since": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_in_osds": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "osd_in_since": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgmap": { 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "pgs_by_state": [], 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pgs": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_pools": 0, 2024-02-18T01:09:15.232 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_objects": 0, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "data_bytes": 0, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_used": 0, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_avail": 0, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "bytes_total": 0 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "fsmap": { 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "by_rank": [], 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "up:standby": 0 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mgrmap": { 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "available": true, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_standbys": 0, 2024-02-18T01:09:15.233 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modules": [ 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "iostat", 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "nfs", 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "restful" 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ], 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "servicemap": { 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 1, 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "modified": "2024-02-18T01:08:57.372205+0000", 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "services": {} 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: }, 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "progress_events": {} 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 >> v1:172.21.15.105:6800/3260915113 conn(0x7f4ca8033740 legacy=0x7f4ca8035be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:15.234 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 >> v1:172.21.15.105:6789/0 conn(0x7f4cc810a0e0 legacy=0x7f4cc810a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:15.235 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 shutdown_connections 2024-02-18T01:09:15.235 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.215+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 >> 172.21.15.105:0/3012413636 conn(0x7f4cc80f6e90 msgr2=0x7f4cc80f7a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:15.235 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.219+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 shutdown_connections 2024-02-18T01:09:15.235 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:15.219+0000 7f4cce32f700 1 -- 172.21.15.105:0/3012413636 wait complete. 2024-02-18T01:09:15.807 INFO:teuthology.orchestra.run.smithi105.stderr:mgr is available 2024-02-18T01:09:16.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:16 smithi105 bash[19734]: cluster 2024-02-18T01:09:15.147017+0000 mon.a (mon.0) 26 : cluster [DBG] mgrmap e4: y(active, since 2s) 2024-02-18T01:09:16.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:16 smithi105 bash[19734]: audit 2024-02-18T01:09:15.218428+0000 mon.a (mon.0) 27 : audit [DBG] from='client.? v1:172.21.15.105:0/3012413636' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-02-18T01:09:16.844 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.835+0000 7f289e850700 1 Processor -- start 2024-02-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.835+0000 7f289e850700 1 -- start start 2024-02-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.835+0000 7f289e850700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f28980fc710 con 0x7f28980fbbe0 2024-02-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.835+0000 7f2897fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28980fbbe0 0x7f28980fc000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57090/0 (socket says 172.21.15.105:57090) 2024-02-18T01:09:16.845 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.835+0000 7f2897fff700 1 -- 172.21.15.105:0/3498465198 learned_addr learned my addr 172.21.15.105:0/3498465198 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3309232235 0 0) 0x7f28980fc710 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28800033f0 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 625985431 0 0) 0x7f28800033f0 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28980fc8a0 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f288c001910 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f288c001fb0 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f2896ffd700 1 -- 172.21.15.105:0/3498465198 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f288c0059b0 con 0x7f28980fbbe0 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- 172.21.15.105:0/3498465198 >> v1:172.21.15.105:6789/0 conn(0x7f28980fbbe0 legacy=0x7f28980fc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- 172.21.15.105:0/3498465198 shutdown_connections 2024-02-18T01:09:16.847 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- 172.21.15.105:0/3498465198 >> 172.21.15.105:0/3498465198 conn(0x7f28980f5530 msgr2=0x7f28980f7950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:16.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- 172.21.15.105:0/3498465198 shutdown_connections 2024-02-18T01:09:16.848 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- 172.21.15.105:0/3498465198 wait complete. 2024-02-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 Processor -- start 2024-02-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- start start 2024-02-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.839+0000 7f289e850700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f289810c980 con 0x7f28980fbbe0 2024-02-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f2897fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f28980fbbe0 0x7f289810c270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57106/0 (socket says 172.21.15.105:57106) 2024-02-18T01:09:16.849 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f2897fff700 1 -- 172.21.15.105:0/3461078083 learned_addr learned my addr 172.21.15.105:0/3461078083 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:16.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285575392 0 0) 0x7f289810c980 con 0x7f28980fbbe0 2024-02-18T01:09:16.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2880005260 con 0x7f28980fbbe0 2024-02-18T01:09:16.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3589003309 0 0) 0x7f2880005260 con 0x7f28980fbbe0 2024-02-18T01:09:16.850 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f289810c980 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f288c001250 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2565450206 0 0) 0x7f289810c980 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289810cb10 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289810cdc0 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2898110ab0 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f288c004da0 con 0x7f28980fbbe0 2024-02-18T01:09:16.851 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f288c0064d0 con 0x7f28980fbbe0 2024-02-18T01:09:16.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 4) v1 ==== 38571+0+0 (unknown 3881454507 0 0) 0x7f288c00fcf0 con 0x7f28980fbbe0 2024-02-18T01:09:16.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.843+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7f288c03fd60 con 0x7f28980fbbe0 2024-02-18T01:09:16.852 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.847+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2884004f50 con 0x7f28980fbbe0 2024-02-18T01:09:16.860 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:16.855+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f288c011380 con 0x7f28980fbbe0 2024-02-18T01:09:17.187 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.179+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f2884005780 con 0x7f28980fbbe0 2024-02-18T01:09:17.189 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f28957fa700 1 -- 172.21.15.105:0/3461078083 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+307 (unknown 1187553405 0 4233090925) 0x7f288c014b50 con 0x7f28980fbbe0 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [global] 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: fsid = 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: log_to_journald = false 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2024-02-18T01:09:17.191 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [mgr] 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2024-02-18T01:09:17.192 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.193 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: [osd] 2024-02-18T01:09:17.193 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2024-02-18T01:09:17.193 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 >> v1:172.21.15.105:6800/3260915113 conn(0x7f28800338e0 legacy=0x7f2880035da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 >> v1:172.21.15.105:6789/0 conn(0x7f28980fbbe0 legacy=0x7f289810c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 shutdown_connections 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 >> 172.21.15.105:0/3461078083 conn(0x7f28980f5530 msgr2=0x7f28980f6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 shutdown_connections 2024-02-18T01:09:17.194 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:17.183+0000 7f289e850700 1 -- 172.21.15.105:0/3461078083 wait complete. 2024-02-18T01:09:17.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:17 smithi105 bash[19734]: audit 2024-02-18T01:09:17.184243+0000 mon.a (mon.0) 28 : audit [INF] from='client.? v1:172.21.15.105:0/3461078083' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-02-18T01:09:17.744 INFO:teuthology.orchestra.run.smithi105.stderr:Enabling cephadm module... 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb2a4c8700 1 Processor -- start 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb2a4c8700 1 -- start start 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb2a4c8700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb240fd890 con 0x7ffb240fcd60 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb23fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb240fcd60 0x7ffb240fd180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57120/0 (socket says 172.21.15.105:57120) 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb23fff700 1 -- 172.21.15.105:0/1452838709 learned_addr learned my addr 172.21.15.105:0/1452838709 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971183197 0 0) 0x7ffb240fd890 con 0x7ffb240fcd60 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb100033f0 con 0x7ffb240fcd60 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 126238773 0 0) 0x7ffb100033f0 con 0x7ffb240fcd60 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb240fda20 con 0x7ffb240fcd60 2024-02-18T01:09:18.800 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb1c001910 con 0x7ffb240fcd60 2024-02-18T01:09:18.801 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ffb1c001fb0 con 0x7ffb240fcd60 2024-02-18T01:09:18.801 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.791+0000 7ffb22ffd700 1 -- 172.21.15.105:0/1452838709 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb1c0059b0 con 0x7ffb240fcd60 2024-02-18T01:09:18.801 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/1452838709 >> v1:172.21.15.105:6789/0 conn(0x7ffb240fcd60 legacy=0x7ffb240fd180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:18.801 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/1452838709 shutdown_connections 2024-02-18T01:09:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/1452838709 >> 172.21.15.105:0/1452838709 conn(0x7ffb240f5520 msgr2=0x7ffb240f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/1452838709 shutdown_connections 2024-02-18T01:09:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/1452838709 wait complete. 2024-02-18T01:09:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 Processor -- start 2024-02-18T01:09:18.802 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- start start 2024-02-18T01:09:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb2a4c8700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb2410a6d0 con 0x7ffb240fcd60 2024-02-18T01:09:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb23fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb240fcd60 0x7ffb24109fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:57124/0 (socket says 172.21.15.105:57124) 2024-02-18T01:09:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb23fff700 1 -- 172.21.15.105:0/548269549 learned_addr learned my addr 172.21.15.105:0/548269549 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3306028539 0 0) 0x7ffb2410a6d0 con 0x7ffb240fcd60 2024-02-18T01:09:18.804 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.795+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb10005260 con 0x7ffb240fcd60 2024-02-18T01:09:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2245202501 0 0) 0x7ffb10005260 con 0x7ffb240fcd60 2024-02-18T01:09:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb2410a6d0 con 0x7ffb240fcd60 2024-02-18T01:09:18.805 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb1c001250 con 0x7ffb240fcd60 2024-02-18T01:09:18.806 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3658667077 0 0) 0x7ffb2410a6d0 con 0x7ffb240fcd60 2024-02-18T01:09:18.806 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2410a860 con 0x7ffb240fcd60 2024-02-18T01:09:18.806 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2410ab10 con 0x7ffb240fcd60 2024-02-18T01:09:18.806 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2410e800 con 0x7ffb240fcd60 2024-02-18T01:09:18.806 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ffb1c004d80 con 0x7ffb240fcd60 2024-02-18T01:09:18.807 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb1c006460 con 0x7ffb240fcd60 2024-02-18T01:09:18.807 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 4) v1 ==== 38571+0+0 (unknown 3881454507 0 0) 0x7ffb1c00fc80 con 0x7ffb240fcd60 2024-02-18T01:09:18.807 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb04004f50 con 0x7ffb240fcd60 2024-02-18T01:09:18.807 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.799+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7ffb1c03ee60 con 0x7ffb240fcd60 2024-02-18T01:09:18.814 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:18.807+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ffb1c011310 con 0x7ffb240fcd60 2024-02-18T01:09:19.222 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.215+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7ffb04005200 con 0x7ffb240fcd60 2024-02-18T01:09:19.276 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.267+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 4013801957 0 0) 0x7ffb1c03e450 con 0x7ffb240fcd60 2024-02-18T01:09:19.279 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.271+0000 7ffb217fa700 1 -- 172.21.15.105:0/548269549 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (unknown 2263024820 0 0) 0x7ffb1c013540 con 0x7ffb240fcd60 2024-02-18T01:09:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 >> v1:172.21.15.105:6800/3260915113 conn(0x7ffb100339e0 legacy=0x7ffb10035ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:19.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 >> v1:172.21.15.105:6789/0 conn(0x7ffb240fcd60 legacy=0x7ffb24109fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:19.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 shutdown_connections 2024-02-18T01:09:19.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 >> 172.21.15.105:0/548269549 conn(0x7ffb240f5520 msgr2=0x7ffb240f6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:19.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 shutdown_connections 2024-02-18T01:09:19.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:19.275+0000 7ffb2a4c8700 1 -- 172.21.15.105:0/548269549 wait complete. 2024-02-18T01:09:19.559 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[19734]: audit 2024-02-18T01:09:19.219434+0000 mon.a (mon.0) 29 : audit [INF] from='client.? v1:172.21.15.105:0/548269549' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-02-18T01:09:19.560 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[20046]: ignoring --setuser ceph since I am not root 2024-02-18T01:09:19.560 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[20046]: ignoring --setgroup ceph since I am not root 2024-02-18T01:09:19.560 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[20046]: debug 2024-02-18T01:09:19.459+0000 7fec08fce680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:09:19.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[20046]: debug 2024-02-18T01:09:19.551+0000 7fec08fce680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:09:20.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:19 smithi105 bash[20046]: debug 2024-02-18T01:09:19.983+0000 7fec08fce680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:09:20.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:20 smithi105 bash[19734]: audit 2024-02-18T01:09:19.276036+0000 mon.a (mon.0) 30 : audit [INF] from='client.? v1:172.21.15.105:0/548269549' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-02-18T01:09:20.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:20 smithi105 bash[19734]: cluster 2024-02-18T01:09:19.276175+0000 mon.a (mon.0) 31 : cluster [DBG] mgrmap e5: y(active, since 6s) 2024-02-18T01:09:21.177 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.167+0000 7ff11fefb700 1 Processor -- start 2024-02-18T01:09:21.177 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.167+0000 7ff11fefb700 1 -- start start 2024-02-18T01:09:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff1180fc110 con 0x7ff11806fc10 2024-02-18T01:09:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11dc97700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff11806fc10 0x7ff118100720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50572/0 (socket says 172.21.15.105:50572) 2024-02-18T01:09:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11dc97700 1 -- 172.21.15.105:0/2365672386 learned_addr learned my addr 172.21.15.105:0/2365672386 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 112851394 0 0) 0x7ff1180fc110 con 0x7ff11806fc10 2024-02-18T01:09:21.178 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1040033f0 con 0x7ff11806fc10 2024-02-18T01:09:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2945028998 0 0) 0x7ff1040033f0 con 0x7ff11806fc10 2024-02-18T01:09:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118102e50 con 0x7ff11806fc10 2024-02-18T01:09:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ff114001590 con 0x7ff11806fc10 2024-02-18T01:09:21.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ff114001c30 con 0x7ff11806fc10 2024-02-18T01:09:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11cc95700 1 -- 172.21.15.105:0/2365672386 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ff1140055d0 con 0x7ff11806fc10 2024-02-18T01:09:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- 172.21.15.105:0/2365672386 >> v1:172.21.15.105:6789/0 conn(0x7ff11806fc10 legacy=0x7ff118100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- 172.21.15.105:0/2365672386 shutdown_connections 2024-02-18T01:09:21.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- 172.21.15.105:0/2365672386 >> 172.21.15.105:0/2365672386 conn(0x7ff1180f5dc0 msgr2=0x7ff1180f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- 172.21.15.105:0/2365672386 shutdown_connections 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- 172.21.15.105:0/2365672386 wait complete. 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 Processor -- start 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.171+0000 7ff11fefb700 1 -- start start 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11fefb700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff11810c900 con 0x7ff11806fc10 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11dc97700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff11806fc10 0x7ff11810c1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50588/0 (socket says 172.21.15.105:50588) 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11dc97700 1 -- 172.21.15.105:0/1976141022 learned_addr learned my addr 172.21.15.105:0/1976141022 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:21.182 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202513171 0 0) 0x7ff11810c900 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff104005260 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 744634532 0 0) 0x7ff104005260 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff11810c900 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ff114000ed0 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4061625270 0 0) 0x7ff11810c900 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11810ca90 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11fefb700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11810cd40 con 0x7ff11806fc10 2024-02-18T01:09:21.183 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11fefb700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff118110a30 con 0x7ff11806fc10 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ff114001490 con 0x7ff11806fc10 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ff1140062f0 con 0x7ff11806fc10 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 4013801957 0 0) 0x7ff11400fbd0 con 0x7ff11806fc10 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7ff11403fcd0 con 0x7ff11806fc10 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff11d496700 1 -- 172.21.15.105:0/1976141022 >> v1:172.21.15.105:6800/3260915113 conn(0x7ff1040339e0 legacy=0x7ff104035ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:21.184 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.175+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0fc004f50 con 0x7ff11806fc10 2024-02-18T01:09:21.193 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.187+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff1140114c0 con 0x7ff11806fc10 2024-02-18T01:09:21.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:20 smithi105 bash[20046]: debug 2024-02-18T01:09:20.887+0000 7fec08fce680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:09:21.238 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:20 smithi105 bash[20046]: debug 2024-02-18T01:09:20.979+0000 7fec08fce680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:09:21.385 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.379+0000 7ff11d496700 1 -- 172.21.15.105:0/1976141022 >> v1:172.21.15.105:6800/3260915113 conn(0x7ff1040339e0 legacy=0x7ff104035ea0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:21.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:21 smithi105 bash[20046]: debug 2024-02-18T01:09:21.231+0000 7fec08fce680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:09:21.576 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.567+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ff0fc005a10 con 0x7ff11806fc10 2024-02-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10effd700 1 -- 172.21.15.105:0/1976141022 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (unknown 3768197548 0 15966916) 0x7ff114014ae0 con 0x7ff11806fc10 2024-02-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:21.577 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 5, 2024-02-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "available": true, 2024-02-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-18T01:09:21.578 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:21.579 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 >> v1:172.21.15.105:6800/3260915113 conn(0x7ff1040339e0 legacy=0x7ff104035ea0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 >> v1:172.21.15.105:6789/0 conn(0x7ff11806fc10 legacy=0x7ff11810c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 shutdown_connections 2024-02-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 >> 172.21.15.105:0/1976141022 conn(0x7ff1180f5dc0 msgr2=0x7ff1180f69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 shutdown_connections 2024-02-18T01:09:21.580 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:21.571+0000 7ff10cff9700 1 -- 172.21.15.105:0/1976141022 wait complete. 2024-02-18T01:09:21.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:21 smithi105 bash[19734]: audit 2024-02-18T01:09:21.573167+0000 mon.a (mon.0) 32 : audit [DBG] from='client.? v1:172.21.15.105:0/1976141022' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T01:09:22.180 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for the mgr to restart... 2024-02-18T01:09:22.180 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mgr epoch 5... 2024-02-18T01:09:22.592 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:22 smithi105 bash[20046]: debug 2024-02-18T01:09:22.331+0000 7fec08fce680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:09:22.592 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:22 smithi105 bash[20046]: debug 2024-02-18T01:09:22.491+0000 7fec08fce680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:22.917 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:22 smithi105 bash[20046]: debug 2024-02-18T01:09:22.583+0000 7fec08fce680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:22.918 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:22 smithi105 bash[20046]: debug 2024-02-18T01:09:22.695+0000 7fec08fce680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:09:23.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:22 smithi105 bash[20046]: debug 2024-02-18T01:09:22.911+0000 7fec08fce680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:09:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9d760f700 1 Processor -- start 2024-02-18T01:09:23.298 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9d760f700 1 -- start start 2024-02-18T01:09:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9d760f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9d00fb0d0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.299 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9d53ab700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa9d00fa5a0 0x7fa9d00fa9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50592/0 (socket says 172.21.15.105:50592) 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9d53ab700 1 -- 172.21.15.105:0/793237436 learned_addr learned my addr 172.21.15.105:0/793237436 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 763395134 0 0) 0x7fa9d00fb0d0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc0033f0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1449669988 0 0) 0x7fa9bc0033f0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d00ffb20 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fa9cc001590 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.300 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fa9cc001c30 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.301 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.291+0000 7fa9c7fff700 1 -- 172.21.15.105:0/793237436 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fa9cc005630 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.301 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/793237436 >> v1:172.21.15.105:6789/0 conn(0x7fa9d00fa5a0 legacy=0x7fa9d00fa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:23.301 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/793237436 shutdown_connections 2024-02-18T01:09:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/793237436 >> 172.21.15.105:0/793237436 conn(0x7fa9d00f5dc0 msgr2=0x7fa9d00f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/793237436 shutdown_connections 2024-02-18T01:09:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/793237436 wait complete. 2024-02-18T01:09:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 Processor -- start 2024-02-18T01:09:23.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- start start 2024-02-18T01:09:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa9d006fc40 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d53ab700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fa9d00fa5a0 0x7fa9d010cd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50602/0 (socket says 172.21.15.105:50602) 2024-02-18T01:09:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d53ab700 1 -- 172.21.15.105:0/133058122 learned_addr learned my addr 172.21.15.105:0/133058122 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:23.303 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956994370 0 0) 0x7fa9d006fc40 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9bc005260 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3496385567 0 0) 0x7fa9bc005260 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.305 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa9d006fc40 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.305 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fa9cc000ed0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.305 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2434397661 0 0) 0x7fa9d006fc40 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.305 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d006fdd0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d010e420 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fa9cc004a00 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.295+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d010e8b0 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.299+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fa9cc006180 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.299+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 5) v1 ==== 38582+0+0 (unknown 4013801957 0 0) 0x7fa9cc007460 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.299+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6800/3260915113 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9bc0375e0 con 0x7fa9bc033910 2024-02-18T01:09:23.306 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.299+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (unknown 738389935 0 0) 0x7fa9cc03fa20 con 0x7fa9d00fa5a0 2024-02-18T01:09:23.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.299+0000 7fa9d4baa700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/3260915113 conn(0x7fa9bc033910 legacy=0x7fa9bc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:23.508 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.503+0000 7fa9d4baa700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/3260915113 conn(0x7fa9bc033910 legacy=0x7fa9bc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:23.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:23 smithi105 bash[20046]: debug 2024-02-18T01:09:23.283+0000 7fec08fce680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:09:23.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:23 smithi105 bash[20046]: debug 2024-02-18T01:09:23.403+0000 7fec08fce680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:23.913 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:23.907+0000 7fa9d4baa700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/3260915113 conn(0x7fa9bc033910 legacy=0x7fa9bc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:24.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:24 smithi105 bash[20046]: debug 2024-02-18T01:09:24.235+0000 7fec08fce680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:09:24.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:24 smithi105 bash[20046]: debug 2024-02-18T01:09:24.335+0000 7fec08fce680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:09:24.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:24 smithi105 bash[20046]: debug 2024-02-18T01:09:24.447+0000 7fec08fce680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:09:24.717 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:24.711+0000 7fa9d4baa700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/3260915113 conn(0x7fa9bc033910 legacy=0x7fa9bc035dd0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3260915113 2024-02-18T01:09:24.947 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:24 smithi105 bash[20046]: debug 2024-02-18T01:09:24.659+0000 7fec08fce680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:09:25.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:24 smithi105 bash[20046]: debug 2024-02-18T01:09:24.939+0000 7fec08fce680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:09:25.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[20046]: debug 2024-02-18T01:09:25.099+0000 7fec08fce680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:09:25.584 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[20046]: debug 2024-02-18T01:09:25.263+0000 7fec08fce680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:25.585 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[20046]: debug 2024-02-18T01:09:25.475+0000 7fec08fce680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:25.603+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mgrmap(e 6) v1 ==== 38441+0+0 (unknown 1151254961 0 0) 0x7fa9cc03f090 con 0x7fa9d00fa5a0 2024-02-18T01:09:25.613 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:25.603+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/3260915113 conn(0x7fa9bc033910 legacy=0x7fa9bc035dd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T01:09:25.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[20046]: debug 2024-02-18T01:09:25.575+0000 7fec08fce680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: cluster 2024-02-18T01:09:25.603661+0000 mon.a (mon.0) 33 : cluster [INF] Active manager daemon y restarted 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: cluster 2024-02-18T01:09:25.604095+0000 mon.a (mon.0) 34 : cluster [INF] Activating manager daemon y 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: cluster 2024-02-18T01:09:25.610475+0000 mon.a (mon.0) 35 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: cluster 2024-02-18T01:09:25.610686+0000 mon.a (mon.0) 36 : cluster [DBG] mgrmap e6: y(active, starting, since 0.00675298s) 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.613177+0000 mon.a (mon.0) 37 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.614984+0000 mon.a (mon.0) 38 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.616427+0000 mon.a (mon.0) 39 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.616841+0000 mon.a (mon.0) 40 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.617234+0000 mon.a (mon.0) 41 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: cluster 2024-02-18T01:09:25.622398+0000 mon.a (mon.0) 42 : cluster [INF] Manager daemon y is now available 2024-02-18T01:09:25.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:25 smithi105 bash[19734]: audit 2024-02-18T01:09:25.644851+0000 mon.a (mon.0) 43 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:26.614 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.607+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mgrmap(e 7) v1 ==== 38476+0+0 (unknown 4024647704 0 0) 0x7fa9cc001890 con 0x7fa9d00fa5a0 2024-02-18T01:09:26.615 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.607+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6800/1813807489 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa9bc0375e0 con 0x7fa9bc038510 2024-02-18T01:09:26.618 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.611+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (unknown 0 0 4198994610) 0x7fa9bc0375e0 con 0x7fa9bc038510 2024-02-18T01:09:26.630 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 --> v1:172.21.15.105:6800/1813807489 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa9d005a310 con 0x7fa9bc038510 2024-02-18T01:09:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9c67fc700 1 -- 172.21.15.105:0/133058122 <== mgr.14116 v1:172.21.15.105:6800/1813807489 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (unknown 0 0 96372106) 0x7fa9d005a310 con 0x7fa9bc038510 2024-02-18T01:09:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:26.631 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2024-02-18T01:09:26.632 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "initialized": true 2024-02-18T01:09:26.632 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:26.632 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6800/1813807489 conn(0x7fa9bc038510 legacy=0x7fa9bc03a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:26.632 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 >> v1:172.21.15.105:6789/0 conn(0x7fa9d00fa5a0 legacy=0x7fa9d010cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:26.633 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 shutdown_connections 2024-02-18T01:09:26.633 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 >> 172.21.15.105:0/133058122 conn(0x7fa9d00f5dc0 msgr2=0x7fa9d00f69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:26.633 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 shutdown_connections 2024-02-18T01:09:26.633 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:26.623+0000 7fa9d760f700 1 -- 172.21.15.105:0/133058122 wait complete. 2024-02-18T01:09:26.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: cephadm 2024-02-18T01:09:25.638997+0000 mgr.y (mgr.14116) 1 : cephadm [INF] Found migration_current of "None". Setting to last migration. 2024-02-18T01:09:26.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.770586+0000 mon.a (mon.0) 44 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:26.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.772226+0000 mon.a (mon.0) 45 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:26.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.775076+0000 mon.a (mon.0) 46 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:26.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.778992+0000 mon.a (mon.0) 47 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:26.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.804896+0000 mon.a (mon.0) 48 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:09:26.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: audit 2024-02-18T01:09:25.809469+0000 mon.a (mon.0) 49 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:09:26.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:26 smithi105 bash[19734]: cluster 2024-02-18T01:09:26.615015+0000 mon.a (mon.0) 50 : cluster [DBG] mgrmap e7: y(active, since 1.01106s) 2024-02-18T01:09:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:mgr epoch 5 is available 2024-02-18T01:09:27.261 INFO:teuthology.orchestra.run.smithi105.stderr:Setting orchestrator backend to cephadm... 2024-02-18T01:09:28.069 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:27 smithi105 bash[19734]: audit 2024-02-18T01:09:26.613271+0000 mgr.y (mgr.14116) 2 : audit [DBG] from='client.14120 v1:172.21.15.105:0/133058122' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T01:09:28.069 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:27 smithi105 bash[19734]: audit 2024-02-18T01:09:26.627312+0000 mgr.y (mgr.14116) 3 : audit [DBG] from='client.14120 v1:172.21.15.105:0/133058122' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb47f74700 1 Processor -- start 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb47f74700 1 -- start start 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb47f74700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb400fc670 con 0x7ffb400fbb40 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb45d10700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb400fbb40 0x7ffb400fbf60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50662/0 (socket says 172.21.15.105:50662) 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb45d10700 1 -- 172.21.15.105:0/1779593596 learned_addr learned my addr 172.21.15.105:0/1779593596 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834168963 0 0) 0x7ffb400fc670 con 0x7ffb400fbb40 2024-02-18T01:09:28.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb280033f0 con 0x7ffb400fbb40 2024-02-18T01:09:28.308 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2106507133 0 0) 0x7ffb280033f0 con 0x7ffb400fbb40 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400ffb80 con 0x7ffb400fbb40 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb30001910 con 0x7ffb400fbb40 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ffb30002080 con 0x7ffb400fbb40 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb44d0e700 1 -- 172.21.15.105:0/1779593596 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb30005a80 con 0x7ffb400fbb40 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.299+0000 7ffb47f74700 1 -- 172.21.15.105:0/1779593596 >> v1:172.21.15.105:6789/0 conn(0x7ffb400fbb40 legacy=0x7ffb400fbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/1779593596 shutdown_connections 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/1779593596 >> 172.21.15.105:0/1779593596 conn(0x7ffb400f5520 msgr2=0x7ffb400f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/1779593596 shutdown_connections 2024-02-18T01:09:28.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/1779593596 wait complete. 2024-02-18T01:09:28.310 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 Processor -- start 2024-02-18T01:09:28.310 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- start start 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb400fa4a0 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb45d10700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb400fbb40 0x7ffb4010cc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50676/0 (socket says 172.21.15.105:50676) 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb45d10700 1 -- 172.21.15.105:0/4135504989 learned_addr learned my addr 172.21.15.105:0/4135504989 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3996005453 0 0) 0x7ffb400fa4a0 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb28005210 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 576433670 0 0) 0x7ffb28005210 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb400fa4a0 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb30001360 con 0x7ffb400fbb40 2024-02-18T01:09:28.312 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3401265430 0 0) 0x7ffb400fa4a0 con 0x7ffb400fbb40 2024-02-18T01:09:28.313 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb400fa630 con 0x7ffb400fbb40 2024-02-18T01:09:28.313 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb4010e3b0 con 0x7ffb400fbb40 2024-02-18T01:09:28.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4010e840 con 0x7ffb400fbb40 2024-02-18T01:09:28.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.303+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7ffb30004f50 con 0x7ffb400fbb40 2024-02-18T01:09:28.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.307+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7ffb300064d0 con 0x7ffb400fbb40 2024-02-18T01:09:28.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.307+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7ffb300017b0 con 0x7ffb400fbb40 2024-02-18T01:09:28.314 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.307+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7ffb3003f100 con 0x7ffb400fbb40 2024-02-18T01:09:28.315 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.307+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c004f50 con 0x7ffb400fbb40 2024-02-18T01:09:28.324 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.315+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ffb30014280 con 0x7ffb400fbb40 2024-02-18T01:09:28.668 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.659+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb2c000bc0 con 0x7ffb28033890 2024-02-18T01:09:28.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.675+0000 7ffb3effd700 1 -- 172.21.15.105:0/4135504989 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7ffb2c000bc0 con 0x7ffb28033890 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 >> v1:172.21.15.105:6800/1813807489 conn(0x7ffb28033890 legacy=0x7ffb28035d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 >> v1:172.21.15.105:6789/0 conn(0x7ffb400fbb40 legacy=0x7ffb4010cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 shutdown_connections 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 >> 172.21.15.105:0/4135504989 conn(0x7ffb400f5520 msgr2=0x7ffb400f6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 shutdown_connections 2024-02-18T01:09:28.686 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:28.679+0000 7ffb47f74700 1 -- 172.21.15.105:0/4135504989 wait complete. 2024-02-18T01:09:28.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:28 smithi105 bash[19734]: cluster 2024-02-18T01:09:27.776216+0000 mon.a (mon.0) 51 : cluster [DBG] mgrmap e8: y(active, since 2s) 2024-02-18T01:09:28.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:28 smithi105 bash[19734]: audit 2024-02-18T01:09:28.676944+0000 mon.a (mon.0) 52 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:28.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:28 smithi105 bash[19734]: audit 2024-02-18T01:09:28.712915+0000 mon.a (mon.0) 53 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:30.048 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:29 smithi105 bash[19734]: audit 2024-02-18T01:09:28.666904+0000 mgr.y (mgr.14116) 4 : audit [DBG] from='client.14126 v1:172.21.15.105:0/4135504989' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 Processor -- start 2024-02-18T01:09:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- start start 2024-02-18T01:09:30.291 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f77740fe3c0 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7779d52700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f77740fd890 0x7f77740fdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35126/0 (socket says 172.21.15.105:35126) 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7779d52700 1 -- 172.21.15.105:0/2042639249 learned_addr learned my addr 172.21.15.105:0/2042639249 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4069619519 0 0) 0x7f77740fe3c0 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c0033f0 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 573771056 0 0) 0x7f775c0033f0 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774102e10 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f776c001910 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f776c002080 con 0x7f77740fd890 2024-02-18T01:09:30.292 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f7778d50700 1 -- 172.21.15.105:0/2042639249 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f776c005a80 con 0x7f77740fd890 2024-02-18T01:09:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- 172.21.15.105:0/2042639249 >> v1:172.21.15.105:6789/0 conn(0x7f77740fd890 legacy=0x7f77740fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- 172.21.15.105:0/2042639249 shutdown_connections 2024-02-18T01:09:30.294 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- 172.21.15.105:0/2042639249 >> 172.21.15.105:0/2042639249 conn(0x7f7774070f30 msgr2=0x7f7774071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- 172.21.15.105:0/2042639249 shutdown_connections 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.283+0000 7f777bfb6700 1 -- 172.21.15.105:0/2042639249 wait complete. 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f777bfb6700 1 Processor -- start 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f777bfb6700 1 -- start start 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f777bfb6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7774072da0 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f7779d52700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f77740fd890 0x7f7774072690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35134/0 (socket says 172.21.15.105:35134) 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f7779d52700 1 -- 172.21.15.105:0/1404161329 learned_addr learned my addr 172.21.15.105:0/1404161329 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219204010 0 0) 0x7f7774072da0 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f775c005210 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3523112014 0 0) 0x7f775c005210 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7774072da0 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f776c004f10 con 0x7f77740fd890 2024-02-18T01:09:30.295 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4161031478 0 0) 0x7f7774072da0 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7774072f30 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7774073180 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7774074f30 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f776c005840 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.287+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f776c006560 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.291+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7f776c0066c0 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.291+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7760004f50 con 0x7f77740fd890 2024-02-18T01:09:30.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.291+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f776c03ee80 con 0x7f77740fd890 2024-02-18T01:09:30.305 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.299+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f776c007df0 con 0x7f77740fd890 2024-02-18T01:09:30.633 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f7760000bc0 con 0x7f775c033950 2024-02-18T01:09:30.634 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f776affd700 1 -- 172.21.15.105:0/1404161329 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (unknown 0 0 2070689548) 0x7f7760000bc0 con 0x7f775c033950 2024-02-18T01:09:30.634 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: value unchanged 2024-02-18T01:09:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 >> v1:172.21.15.105:6800/1813807489 conn(0x7f775c033950 legacy=0x7f775c035e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:30.636 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 >> v1:172.21.15.105:6789/0 conn(0x7f77740fd890 legacy=0x7f7774072690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 shutdown_connections 2024-02-18T01:09:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.627+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 >> 172.21.15.105:0/1404161329 conn(0x7f7774070f30 msgr2=0x7f77740facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.631+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 shutdown_connections 2024-02-18T01:09:30.637 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:30.631+0000 7f777bfb6700 1 -- 172.21.15.105:0/1404161329 wait complete. 2024-02-18T01:09:31.223 INFO:teuthology.orchestra.run.smithi105.stderr:Generating ssh key... 2024-02-18T01:09:31.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:31 smithi105 bash[19734]: audit 2024-02-18T01:09:30.630098+0000 mgr.y (mgr.14116) 5 : audit [DBG] from='client.14128 v1:172.21.15.105:0/1404161329' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:32.334 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f9246c08700 1 Processor -- start 2024-02-18T01:09:32.334 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f9246c08700 1 -- start start 2024-02-18T01:09:32.334 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f9246c08700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f92400fe3c0 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f92449a4700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92400fd890 0x7f92400fdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35150/0 (socket says 172.21.15.105:35150) 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f92449a4700 1 -- 172.21.15.105:0/41098352 learned_addr learned my addr 172.21.15.105:0/41098352 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 741079749 0 0) 0x7f92400fe3c0 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92280033f0 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014000692 0 0) 0x7f92280033f0 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240102e10 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f9234001590 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f9234001d00 con 0x7f92400fd890 2024-02-18T01:09:32.336 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.327+0000 7f923f7fe700 1 -- 172.21.15.105:0/41098352 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f9234005700 con 0x7f92400fd890 2024-02-18T01:09:32.337 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- 172.21.15.105:0/41098352 >> v1:172.21.15.105:6789/0 conn(0x7f92400fd890 legacy=0x7f92400fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:32.337 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- 172.21.15.105:0/41098352 shutdown_connections 2024-02-18T01:09:32.337 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- 172.21.15.105:0/41098352 >> 172.21.15.105:0/41098352 conn(0x7f9240070f30 msgr2=0x7f9240071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:32.337 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- 172.21.15.105:0/41098352 shutdown_connections 2024-02-18T01:09:32.337 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- 172.21.15.105:0/41098352 wait complete. 2024-02-18T01:09:32.338 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 Processor -- start 2024-02-18T01:09:32.338 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- start start 2024-02-18T01:09:32.339 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f9246c08700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f924010c8f0 con 0x7f92400fd890 2024-02-18T01:09:32.339 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f92449a4700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f92400fd890 0x7f924010c1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35166/0 (socket says 172.21.15.105:35166) 2024-02-18T01:09:32.339 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f92449a4700 1 -- 172.21.15.105:0/2133383808 learned_addr learned my addr 172.21.15.105:0/2133383808 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70518303 0 0) 0x7f924010c8f0 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9228005210 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2278598648 0 0) 0x7f9228005210 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f924010c8f0 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f9234004b90 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.331+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 651915845 0 0) 0x7f924010c8f0 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924010ca80 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924010cd30 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9240110a20 con 0x7f92400fd890 2024-02-18T01:09:32.341 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f92340054c0 con 0x7f92400fd890 2024-02-18T01:09:32.342 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f92340061e0 con 0x7f92400fd890 2024-02-18T01:09:32.342 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7f9234006340 con 0x7f92400fd890 2024-02-18T01:09:32.342 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f922c004f50 con 0x7f92400fd890 2024-02-18T01:09:32.342 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.335+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f923403fa10 con 0x7f92400fd890 2024-02-18T01:09:32.349 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.343+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f9234007a70 con 0x7f92400fd890 2024-02-18T01:09:32.676 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.667+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f922c000bc0 con 0x7f9228033950 2024-02-18T01:09:32.920 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.911+0000 7f923dffb700 1 -- 172.21.15.105:0/2133383808 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f922c000bc0 con 0x7f9228033950 2024-02-18T01:09:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 >> v1:172.21.15.105:6800/1813807489 conn(0x7f9228033950 legacy=0x7f9228035e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 >> v1:172.21.15.105:6789/0 conn(0x7f92400fd890 legacy=0x7f924010c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:32.923 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 shutdown_connections 2024-02-18T01:09:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 >> 172.21.15.105:0/2133383808 conn(0x7f9240070f30 msgr2=0x7f92400facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 shutdown_connections 2024-02-18T01:09:32.924 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:32.915+0000 7f9246c08700 1 -- 172.21.15.105:0/2133383808 wait complete. 2024-02-18T01:09:33.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: Generating public/private rsa key pair. 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: Your identification has been saved in /tmp/tmpxz50x4ns/key. 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: Your public key has been saved in /tmp/tmpxz50x4ns/key.pub. 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: The key fingerprint is: 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: SHA256:BaHSkfN2/i1ZhWnC2mNg0Fpfq9whg0sFoKry25OdGdo ceph-358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: The key's randomart image is: 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: +---[RSA 3072]----+ 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | ..++.. | 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | .o+..o . . | 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | . +o +.= . + | 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | o +.* * * .| 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | . .S= * B o | 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | . . + * o | 2024-02-18T01:09:33.243 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: |. . = + o = | 2024-02-18T01:09:33.244 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | o .+ E + . | 2024-02-18T01:09:33.244 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: | o... . | 2024-02-18T01:09:33.244 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:32 smithi105 bash[20046]: +----[SHA256]-----+ 2024-02-18T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:33 smithi105 bash[19734]: audit 2024-02-18T01:09:32.672940+0000 mgr.y (mgr.14116) 6 : audit [DBG] from='client.14130 v1:172.21.15.105:0/2133383808' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:33 smithi105 bash[19734]: cephadm 2024-02-18T01:09:32.673388+0000 mgr.y (mgr.14116) 7 : cephadm [INF] Generating ssh key... 2024-02-18T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:33 smithi105 bash[19734]: audit 2024-02-18T01:09:32.910163+0000 mon.a (mon.0) 54 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:33 smithi105 bash[19734]: audit 2024-02-18T01:09:32.914645+0000 mon.a (mon.0) 55 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:34.546 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 Processor -- start 2024-02-18T01:09:34.546 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- start start 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f674c0fe3e0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674b7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f674c0fd8b0 0x7f674c0fdcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35182/0 (socket says 172.21.15.105:35182) 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674b7fe700 1 -- 172.21.15.105:0/4177189124 learned_addr learned my addr 172.21.15.105:0/4177189124 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308545907 0 0) 0x7f674c0fe3e0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67380033f0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118057092 0 0) 0x7f67380033f0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.547 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c102e30 con 0x7f674c0fd8b0 2024-02-18T01:09:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f673c001910 con 0x7f674c0fd8b0 2024-02-18T01:09:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f673c002080 con 0x7f674c0fd8b0 2024-02-18T01:09:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f674a7fc700 1 -- 172.21.15.105:0/4177189124 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f673c005a80 con 0x7f674c0fd8b0 2024-02-18T01:09:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- 172.21.15.105:0/4177189124 >> v1:172.21.15.105:6789/0 conn(0x7f674c0fd8b0 legacy=0x7f674c0fdcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:34.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- 172.21.15.105:0/4177189124 shutdown_connections 2024-02-18T01:09:34.549 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- 172.21.15.105:0/4177189124 >> 172.21.15.105:0/4177189124 conn(0x7f674c070ef0 msgr2=0x7f674c071300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:34.549 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- 172.21.15.105:0/4177189124 shutdown_connections 2024-02-18T01:09:34.549 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.539+0000 7f6752020700 1 -- 172.21.15.105:0/4177189124 wait complete. 2024-02-18T01:09:34.550 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6752020700 1 Processor -- start 2024-02-18T01:09:34.550 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6752020700 1 -- start start 2024-02-18T01:09:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6752020700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f674c10c930 con 0x7f674c0fd8b0 2024-02-18T01:09:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f674b7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f674c0fd8b0 0x7f674c10c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35184/0 (socket says 172.21.15.105:35184) 2024-02-18T01:09:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f674b7fe700 1 -- 172.21.15.105:0/4052703903 learned_addr learned my addr 172.21.15.105:0/4052703903 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:34.551 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1195360092 0 0) 0x7f674c10c930 con 0x7f674c0fd8b0 2024-02-18T01:09:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6738005210 con 0x7f674c0fd8b0 2024-02-18T01:09:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2751565882 0 0) 0x7f6738005210 con 0x7f674c0fd8b0 2024-02-18T01:09:34.552 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f674c10c930 con 0x7f674c0fd8b0 2024-02-18T01:09:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f673c004f10 con 0x7f674c0fd8b0 2024-02-18T01:09:34.553 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1671203325 0 0) 0x7f674c10c930 con 0x7f674c0fd8b0 2024-02-18T01:09:34.555 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c10cac0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.555 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.543+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674c10cd70 con 0x7f674c0fd8b0 2024-02-18T01:09:34.556 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f674c110a60 con 0x7f674c0fd8b0 2024-02-18T01:09:34.556 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f673c001560 con 0x7f674c0fd8b0 2024-02-18T01:09:34.556 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f673c006410 con 0x7f674c0fd8b0 2024-02-18T01:09:34.556 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f672c004f50 con 0x7f674c0fd8b0 2024-02-18T01:09:34.556 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7f673c00fc50 con 0x7f674c0fd8b0 2024-02-18T01:09:34.557 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.547+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f673c03eec0 con 0x7f674c0fd8b0 2024-02-18T01:09:34.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.555+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f673c011330 con 0x7f674c0fd8b0 2024-02-18T01:09:34.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.875+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f672c000bc0 con 0x7f6738033940 2024-02-18T01:09:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.875+0000 7f6748ff9700 1 -- 172.21.15.105:0/4052703903 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (unknown 0 0 2053912186) 0x7f672c000bc0 con 0x7f6738033940 2024-02-18T01:09:34.884 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnpuppM8DyREG3Wu9GmGHavHGZklhdLksASVSW3iktRVra0xiGXTConE4UIwvCwTmFDDv30gNMHlRowWKW9LbtA+txqSifYM/Zw1m47KB8pzwOz+TictO9Khg1pYY5Wjra0fJc1NBh9RkXNMMK2g+o4f93g4q7fix9vjbrXEdFF3BXatOIEt1AoqbmH3lka7YTgHGA2u7tTGdOoij2CLbWsE3zU/7Ec+WfDOhKPI+Sqlq24ASlZtPr2J0X/RRPG/4GAc4NA1N5A3XQC0sg1bVsSFNBwEK3dXyJTl+vXAqeRqCCi2wrCdwVak3Qo22buB+FA41L7vplBGMLliovYnQLLZahHlHik4q4/4MCCnOCKFzMu47yMj9glrhE1xCYK6aaSU8fQQJjZ9Ad6uJlwG7ha7s81rfOuyELFojeQl/sYMG/KeXXym1hnRQls55wA91I/8j5yQKrE7wDdCWaX6YwPmvBGAsrUi7FWKuGtGYZLTh2c2E9mmJzmJghKxjtPkk= ceph-358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:09:34.886 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 >> v1:172.21.15.105:6800/1813807489 conn(0x7f6738033940 legacy=0x7f6738035e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 >> v1:172.21.15.105:6789/0 conn(0x7f674c0fd8b0 legacy=0x7f674c10c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 shutdown_connections 2024-02-18T01:09:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 >> 172.21.15.105:0/4052703903 conn(0x7f674c070ef0 msgr2=0x7f674c0facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 shutdown_connections 2024-02-18T01:09:34.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:34.879+0000 7f6752020700 1 -- 172.21.15.105:0/4052703903 wait complete. 2024-02-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi105.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-02-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi105.stderr:Adding key to root@localhost authorized_keys... 2024-02-18T01:09:35.477 INFO:teuthology.orchestra.run.smithi105.stderr:Adding host smithi105... 2024-02-18T01:09:35.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:35 smithi105 bash[19734]: audit 2024-02-18T01:09:34.880572+0000 mgr.y (mgr.14116) 8 : audit [DBG] from='client.14132 v1:172.21.15.105:0/4052703903' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01ad939700 1 Processor -- start 2024-02-18T01:09:36.619 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01ad939700 1 -- start start 2024-02-18T01:09:36.620 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01ad939700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f01a80ffaf0 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a6ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f01a806fb50 0x7f01a80fe3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35200/0 (socket says 172.21.15.105:35200) 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a6ffd700 1 -- 172.21.15.105:0/3134771657 learned_addr learned my addr 172.21.15.105:0/3134771657 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2386996961 0 0) 0x7f01a80ffaf0 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01940033f0 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 973505848 0 0) 0x7f01940033f0 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a8100c90 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0198001910 con 0x7f01a806fb50 2024-02-18T01:09:36.621 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.611+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f0198001fb0 con 0x7f01a806fb50 2024-02-18T01:09:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01a5ffb700 1 -- 172.21.15.105:0/3134771657 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f01980059b0 con 0x7f01a806fb50 2024-02-18T01:09:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/3134771657 >> v1:172.21.15.105:6789/0 conn(0x7f01a806fb50 legacy=0x7f01a80fe3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:36.622 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/3134771657 shutdown_connections 2024-02-18T01:09:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/3134771657 >> 172.21.15.105:0/3134771657 conn(0x7f01a80f5d90 msgr2=0x7f01a80f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/3134771657 shutdown_connections 2024-02-18T01:09:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/3134771657 wait complete. 2024-02-18T01:09:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 Processor -- start 2024-02-18T01:09:36.623 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- start start 2024-02-18T01:09:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f01a80fbde0 con 0x7f01a806fb50 2024-02-18T01:09:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01a6ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f01a806fb50 0x7f01a80fa2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35202/0 (socket says 172.21.15.105:35202) 2024-02-18T01:09:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01a6ffd700 1 -- 172.21.15.105:0/680470314 learned_addr learned my addr 172.21.15.105:0/680470314 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:36.624 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193585762 0 0) 0x7f01a80fbde0 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0194005260 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2057552280 0 0) 0x7f0194005260 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f01a80fbde0 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0198001250 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1184012231 0 0) 0x7f01a80fbde0 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a80fbf70 con 0x7f01a806fb50 2024-02-18T01:09:36.625 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.615+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80fc220 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80face0 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7f0198001bb0 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f01980064d0 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0188004f50 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7f019800fdb0 con 0x7f01a806fb50 2024-02-18T01:09:36.626 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.619+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f019803ef30 con 0x7f01a806fb50 2024-02-18T01:09:36.638 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.631+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f01980113e0 con 0x7f01a806fb50 2024-02-18T01:09:36.962 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:36.955+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}) v1 -- 0x7f0188000bc0 con 0x7f01940337a0 2024-02-18T01:09:37.377 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:37 smithi105 bash[20046]: Warning: Permanently added '172.21.15.105' (ECDSA) to the list of known hosts. 2024-02-18T01:09:37.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:37 smithi105 bash[19734]: audit 2024-02-18T01:09:36.959487+0000 mgr.y (mgr.14116) 9 : audit [DBG] from='client.14134 v1:172.21.15.105:0/680470314' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi105", "addr": "172.21.15.105", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.751+0000 7f019ffff700 1 -- 172.21.15.105:0/680470314 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2555400861) 0x7f0188000bc0 con 0x7f01940337a0 2024-02-18T01:09:37.757 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: Added host 'smithi105' with addr '172.21.15.105' 2024-02-18T01:09:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 >> v1:172.21.15.105:6800/1813807489 conn(0x7f01940337a0 legacy=0x7f0194035c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 >> v1:172.21.15.105:6789/0 conn(0x7f01a806fb50 legacy=0x7f01a80fa2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 shutdown_connections 2024-02-18T01:09:37.761 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 >> 172.21.15.105:0/680470314 conn(0x7f01a80f5d90 msgr2=0x7f01a80fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 shutdown_connections 2024-02-18T01:09:37.762 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:37.755+0000 7f01ad939700 1 -- 172.21.15.105:0/680470314 wait complete. 2024-02-18T01:09:38.559 INFO:teuthology.orchestra.run.smithi105.stderr:Deploying unmanaged mon service... 2024-02-18T01:09:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:38 smithi105 bash[19734]: audit 2024-02-18T01:09:37.752706+0000 mon.a (mon.0) 56 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:38 smithi105 bash[19734]: cephadm 2024-02-18T01:09:37.753344+0000 mgr.y (mgr.14116) 10 : cephadm [INF] Added host smithi105 2024-02-18T01:09:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:38 smithi105 bash[19734]: audit 2024-02-18T01:09:37.803300+0000 mon.a (mon.0) 57 : audit [DBG] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 Processor -- start 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 -- start start 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe3480fb0b0 con 0x7fe3480fa580 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe347fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3480fa580 0x7fe3480fa9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48550/0 (socket says 172.21.15.105:48550) 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe347fff700 1 -- 172.21.15.105:0/3687997614 learned_addr learned my addr 172.21.15.105:0/3687997614 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3983488640 0 0) 0x7fe3480fb0b0 con 0x7fe3480fa580 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3300033f0 con 0x7fe3480fa580 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3511314240 0 0) 0x7fe3300033f0 con 0x7fe3480fa580 2024-02-18T01:09:40.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480ffb00 con 0x7fe3480fa580 2024-02-18T01:09:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe338001590 con 0x7fe3480fa580 2024-02-18T01:09:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fe338001d00 con 0x7fe3480fa580 2024-02-18T01:09:40.124 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe346ffd700 1 -- 172.21.15.105:0/3687997614 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe338005700 con 0x7fe3480fa580 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 -- 172.21.15.105:0/3687997614 >> v1:172.21.15.105:6789/0 conn(0x7fe3480fa580 legacy=0x7fe3480fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 -- 172.21.15.105:0/3687997614 shutdown_connections 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.115+0000 7fe34e78b700 1 -- 172.21.15.105:0/3687997614 >> 172.21.15.105:0/3687997614 conn(0x7fe3480f5dc0 msgr2=0x7fe3480f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- 172.21.15.105:0/3687997614 shutdown_connections 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- 172.21.15.105:0/3687997614 wait complete. 2024-02-18T01:09:40.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 Processor -- start 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- start start 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe348109670 con 0x7fe3480fa580 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe347fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe3480fa580 0x7fe348108f60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48562/0 (socket says 172.21.15.105:48562) 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe347fff700 1 -- 172.21.15.105:0/2917490758 learned_addr learned my addr 172.21.15.105:0/2917490758 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165314058 0 0) 0x7fe348109670 con 0x7fe3480fa580 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe330005210 con 0x7fe3480fa580 2024-02-18T01:09:40.127 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 643315898 0 0) 0x7fe330005210 con 0x7fe3480fa580 2024-02-18T01:09:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe348109670 con 0x7fe3480fa580 2024-02-18T01:09:40.128 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe338004b90 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 769518567 0 0) 0x7fe348109670 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348109800 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe348109ab0 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34810d7a0 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1036+0+0 (unknown 3619383151 0 0) 0x7fe338001180 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.119+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe338006030 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.123+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7fe33800f910 con 0x7fe3480fa580 2024-02-18T01:09:40.129 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.123+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe334004f50 con 0x7fe3480fa580 2024-02-18T01:09:40.130 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.123+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7fe33803ec50 con 0x7fe3480fa580 2024-02-18T01:09:40.135 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.127+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fe338011040 con 0x7fe3480fa580 2024-02-18T01:09:40.485 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.479+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe334000bc0 con 0x7fe3300338e0 2024-02-18T01:09:40.491 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.483+0000 7fe3457fa700 1 -- 172.21.15.105:0/2917490758 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7fe334000bc0 con 0x7fe3300338e0 2024-02-18T01:09:40.492 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: Scheduled mon update... 2024-02-18T01:09:40.494 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 >> v1:172.21.15.105:6800/1813807489 conn(0x7fe3300338e0 legacy=0x7fe330035da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:40.494 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 >> v1:172.21.15.105:6789/0 conn(0x7fe3480fa580 legacy=0x7fe348108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:40.494 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 shutdown_connections 2024-02-18T01:09:40.495 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 >> 172.21.15.105:0/2917490758 conn(0x7fe3480f5dc0 msgr2=0x7fe3480f69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:40.495 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 shutdown_connections 2024-02-18T01:09:40.495 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:40.487+0000 7fe34e78b700 1 -- 172.21.15.105:0/2917490758 wait complete. 2024-02-18T01:09:41.101 INFO:teuthology.orchestra.run.smithi105.stderr:Deploying unmanaged mgr service... 2024-02-18T01:09:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:41 smithi105 bash[19734]: audit 2024-02-18T01:09:40.482771+0000 mgr.y (mgr.14116) 11 : audit [DBG] from='client.14136 v1:172.21.15.105:0/2917490758' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:41 smithi105 bash[19734]: cephadm 2024-02-18T01:09:40.484862+0000 mgr.y (mgr.14116) 12 : cephadm [INF] Saving service mon spec with placement count:5 2024-02-18T01:09:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:41 smithi105 bash[19734]: audit 2024-02-18T01:09:40.487573+0000 mon.a (mon.0) 58 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:41 smithi105 bash[19734]: audit 2024-02-18T01:09:40.638534+0000 mon.a (mon.0) 59 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:41 smithi105 bash[19734]: audit 2024-02-18T01:09:40.932402+0000 mon.a (mon.0) 60 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7eb13c700 1 Processor -- start 2024-02-18T01:09:42.151 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7eb13c700 1 -- start start 2024-02-18T01:09:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7eb13c700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7e40fd7b0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7e8ed8700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb7e40fcc80 0x7fb7e40fd0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48566/0 (socket says 172.21.15.105:48566) 2024-02-18T01:09:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7e8ed8700 1 -- 172.21.15.105:0/1997821493 learned_addr learned my addr 172.21.15.105:0/1997821493 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071437981 0 0) 0x7fb7e40fd7b0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.152 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.143+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d40033f0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 24966413 0 0) 0x7fb7d40033f0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40fd940 con 0x7fb7e40fcc80 2024-02-18T01:09:42.153 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb7d0001590 con 0x7fb7e40fcc80 2024-02-18T01:09:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb7d0001c50 con 0x7fb7e40fcc80 2024-02-18T01:09:42.154 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7e37fe700 1 -- 172.21.15.105:0/1997821493 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb7d0005670 con 0x7fb7e40fcc80 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- 172.21.15.105:0/1997821493 >> v1:172.21.15.105:6789/0 conn(0x7fb7e40fcc80 legacy=0x7fb7e40fd0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- 172.21.15.105:0/1997821493 shutdown_connections 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- 172.21.15.105:0/1997821493 >> 172.21.15.105:0/1997821493 conn(0x7fb7e40f5520 msgr2=0x7fb7e40f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- 172.21.15.105:0/1997821493 shutdown_connections 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- 172.21.15.105:0/1997821493 wait complete. 2024-02-18T01:09:42.155 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 Processor -- start 2024-02-18T01:09:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.147+0000 7fb7eb13c700 1 -- start start 2024-02-18T01:09:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7eb13c700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7e40fc4e0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.156 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e8ed8700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb7e40fcc80 0x7fb7e40fa9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48574/0 (socket says 172.21.15.105:48574) 2024-02-18T01:09:42.157 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e8ed8700 1 -- 172.21.15.105:0/2996205003 learned_addr learned my addr 172.21.15.105:0/2996205003 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1688712844 0 0) 0x7fb7e40fc4e0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d4005260 con 0x7fb7e40fcc80 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 821569928 0 0) 0x7fb7d4005260 con 0x7fb7e40fcc80 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7e40fc4e0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb7d0000ff0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.158 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2634096121 0 0) 0x7fb7e40fc4e0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7e40fc670 con 0x7fb7e40fcc80 2024-02-18T01:09:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7e40fc920 con 0x7fb7e40fcc80 2024-02-18T01:09:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7e40fb3e0 con 0x7fb7e40fcc80 2024-02-18T01:09:42.159 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb7d0004800 con 0x7fb7e40fcc80 2024-02-18T01:09:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb7d0006190 con 0x7fb7e40fcc80 2024-02-18T01:09:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7fb7d000fa90 con 0x7fb7e40fcc80 2024-02-18T01:09:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7fb7d003fb60 con 0x7fb7e40fcc80 2024-02-18T01:09:42.160 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.151+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c8004f50 con 0x7fb7e40fcc80 2024-02-18T01:09:42.168 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.159+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb7d003fe60 con 0x7fb7e40fcc80 2024-02-18T01:09:42.512 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.503+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 --> v1:172.21.15.105:6800/1813807489 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fb7c8000bc0 con 0x7fb7d40337d0 2024-02-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.511+0000 7fb7e1ffb700 1 -- 172.21.15.105:0/2996205003 <== mgr.14116 v1:172.21.15.105:6800/1813807489 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fb7c8000bc0 con 0x7fb7d40337d0 2024-02-18T01:09:42.520 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: Scheduled mgr update... 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 >> v1:172.21.15.105:6800/1813807489 conn(0x7fb7d40337d0 legacy=0x7fb7d4035c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 >> v1:172.21.15.105:6789/0 conn(0x7fb7e40fcc80 legacy=0x7fb7e40fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 shutdown_connections 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 >> 172.21.15.105:0/2996205003 conn(0x7fb7e40f5520 msgr2=0x7fb7e40f6120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 shutdown_connections 2024-02-18T01:09:42.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:42.515+0000 7fb7eb13c700 1 -- 172.21.15.105:0/2996205003 wait complete. 2024-02-18T01:09:43.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:43 smithi105 bash[19734]: audit 2024-02-18T01:09:42.510361+0000 mgr.y (mgr.14116) 13 : audit [DBG] from='client.14138 v1:172.21.15.105:0/2996205003' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:09:43.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:43 smithi105 bash[19734]: cephadm 2024-02-18T01:09:42.511519+0000 mgr.y (mgr.14116) 14 : cephadm [INF] Saving service mgr spec with placement count:2 2024-02-18T01:09:43.847 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:43 smithi105 bash[19734]: audit 2024-02-18T01:09:42.515945+0000 mon.a (mon.0) 61 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.195+0000 7faec998d700 1 Processor -- start 2024-02-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.195+0000 7faec998d700 1 -- start start 2024-02-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.195+0000 7faec998d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faec4072090 con 0x7faec4071c70 2024-02-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec2ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faec4071c70 0x7faec4070210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48588/0 (socket says 172.21.15.105:48588) 2024-02-18T01:09:44.205 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec2ffd700 1 -- 172.21.15.105:0/3157454620 learned_addr learned my addr 172.21.15.105:0/3157454620 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1757747396 0 0) 0x7faec4072090 con 0x7faec4071c70 2024-02-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea80033f0 con 0x7faec4071c70 2024-02-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3120884665 0 0) 0x7faea80033f0 con 0x7faec4071c70 2024-02-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec4070920 con 0x7faec4071c70 2024-02-18T01:09:44.206 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7faeb0001590 con 0x7faec4071c70 2024-02-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7faeb0001d20 con 0x7faec4071c70 2024-02-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec1ffb700 1 -- 172.21.15.105:0/3157454620 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7faeb0005740 con 0x7faec4071c70 2024-02-18T01:09:44.207 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec998d700 1 -- 172.21.15.105:0/3157454620 >> v1:172.21.15.105:6789/0 conn(0x7faec4071c70 legacy=0x7faec4070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec998d700 1 -- 172.21.15.105:0/3157454620 shutdown_connections 2024-02-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec998d700 1 -- 172.21.15.105:0/3157454620 >> 172.21.15.105:0/3157454620 conn(0x7faec40f6e90 msgr2=0x7faec40f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:44.208 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec998d700 1 -- 172.21.15.105:0/3157454620 shutdown_connections 2024-02-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.199+0000 7faec998d700 1 -- 172.21.15.105:0/3157454620 wait complete. 2024-02-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec998d700 1 Processor -- start 2024-02-18T01:09:44.209 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec998d700 1 -- start start 2024-02-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec998d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faec410c920 con 0x7faec4071c70 2024-02-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec2ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faec4071c70 0x7faec410c210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48596/0 (socket says 172.21.15.105:48596) 2024-02-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec2ffd700 1 -- 172.21.15.105:0/273195087 learned_addr learned my addr 172.21.15.105:0/273195087 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363635601 0 0) 0x7faec410c920 con 0x7faec4071c70 2024-02-18T01:09:44.210 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea8005210 con 0x7faec4071c70 2024-02-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4137053931 0 0) 0x7faea8005210 con 0x7faec4071c70 2024-02-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faec410c920 con 0x7faec4071c70 2024-02-18T01:09:44.211 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7faeb0004870 con 0x7faec4071c70 2024-02-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1809557640 0 0) 0x7faec410c920 con 0x7faec4071c70 2024-02-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec410cab0 con 0x7faec4071c70 2024-02-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec410cd60 con 0x7faec4071c70 2024-02-18T01:09:44.212 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faec4110a50 con 0x7faec4071c70 2024-02-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7faeb00011c0 con 0x7faec4071c70 2024-02-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7faeb0006180 con 0x7faec4071c70 2024-02-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7faeb000fa80 con 0x7faec4071c70 2024-02-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.203+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7faeb003fc80 con 0x7faec4071c70 2024-02-18T01:09:44.213 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.207+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeac004f50 con 0x7faec4071c70 2024-02-18T01:09:44.218 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.211+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7faeb0011380 con 0x7faec4071c70 2024-02-18T01:09:44.542 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.535+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7faeac005200 con 0x7faec4071c70 2024-02-18T01:09:44.548 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.539+0000 7faebbfff700 1 -- 172.21.15.105:0/273195087 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (unknown 2684808024 0 0) 0x7faeb0013270 con 0x7faec4071c70 2024-02-18T01:09:44.553 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.543+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 >> v1:172.21.15.105:6800/1813807489 conn(0x7faea8033840 legacy=0x7faea8035d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:44.553 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.547+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 >> v1:172.21.15.105:6789/0 conn(0x7faec4071c70 legacy=0x7faec410c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.547+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 shutdown_connections 2024-02-18T01:09:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.547+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 >> 172.21.15.105:0/273195087 conn(0x7faec40f6e90 msgr2=0x7faec40ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.547+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 shutdown_connections 2024-02-18T01:09:44.554 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:44.547+0000 7faec998d700 1 -- 172.21.15.105:0/273195087 wait complete. 2024-02-18T01:09:45.799 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:45 smithi105 bash[19734]: audit 2024-02-18T01:09:44.544335+0000 mon.a (mon.0) 62 : audit [INF] from='client.? v1:172.21.15.105:0/273195087' entity='client.admin' 2024-02-18T01:09:46.117 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 Processor -- start 2024-02-18T01:09:46.117 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- start start 2024-02-18T01:09:46.117 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe6900fc190 con 0x7fe6900fb660 2024-02-18T01:09:46.118 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe6900fb660 0x7fe6900fba80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48604/0 (socket says 172.21.15.105:48604) 2024-02-18T01:09:46.118 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68effd700 1 -- 172.21.15.105:0/72369237 learned_addr learned my addr 172.21.15.105:0/72369237 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:46.118 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 728675488 0 0) 0x7fe6900fc190 con 0x7fe6900fb660 2024-02-18T01:09:46.118 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67c0033f0 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127828351 0 0) 0x7fe67c0033f0 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690100be0 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe688001590 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fe688001d20 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe68dffb700 1 -- 172.21.15.105:0/72369237 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe6880057a0 con 0x7fe6900fb660 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- 172.21.15.105:0/72369237 >> v1:172.21.15.105:6789/0 conn(0x7fe6900fb660 legacy=0x7fe6900fba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- 172.21.15.105:0/72369237 shutdown_connections 2024-02-18T01:09:46.119 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- 172.21.15.105:0/72369237 >> 172.21.15.105:0/72369237 conn(0x7fe6900f6ea0 msgr2=0x7fe6900f92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:46.120 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- 172.21.15.105:0/72369237 shutdown_connections 2024-02-18T01:09:46.120 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- 172.21.15.105:0/72369237 wait complete. 2024-02-18T01:09:46.120 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 Processor -- start 2024-02-18T01:09:46.120 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.110+0000 7fe69560f700 1 -- start start 2024-02-18T01:09:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe69560f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fe690071d00 con 0x7fe6900fb660 2024-02-18T01:09:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe68effd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fe6900fb660 0x7fe6900701f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48608/0 (socket says 172.21.15.105:48608) 2024-02-18T01:09:46.121 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe68effd700 1 -- 172.21.15.105:0/3903104221 learned_addr learned my addr 172.21.15.105:0/3903104221 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358358068 0 0) 0x7fe690071d00 con 0x7fe6900fb660 2024-02-18T01:09:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe67c005210 con 0x7fe6900fb660 2024-02-18T01:09:46.122 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3222179829 0 0) 0x7fe67c005210 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fe690071d00 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe6880057a0 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3299173225 0 0) 0x7fe690071d00 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690071e90 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe690072140 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fe690070c00 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fe688004910 con 0x7fe6900fb660 2024-02-18T01:09:46.123 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe688006190 con 0x7fe6900fb660 2024-02-18T01:09:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7fe68800fa70 con 0x7fe6900fb660 2024-02-18T01:09:46.124 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.114+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7fe68803ed60 con 0x7fe6900fb660 2024-02-18T01:09:46.125 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.118+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe670004f50 con 0x7fe6900fb660 2024-02-18T01:09:46.130 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.122+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fe688014cc0 con 0x7fe6900fb660 2024-02-18T01:09:46.455 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.446+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe670005780 con 0x7fe6900fb660 2024-02-18T01:09:46.458 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.450+0000 7fe687fff700 1 -- 172.21.15.105:0/3903104221 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (unknown 1496729260 0 0) 0x7fe688013130 con 0x7fe6900fb660 2024-02-18T01:09:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.454+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 >> v1:172.21.15.105:6800/1813807489 conn(0x7fe67c033840 legacy=0x7fe67c035d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:46.464 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.454+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 >> v1:172.21.15.105:6789/0 conn(0x7fe6900fb660 legacy=0x7fe6900701f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.454+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 shutdown_connections 2024-02-18T01:09:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.454+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 >> 172.21.15.105:0/3903104221 conn(0x7fe6900f6ea0 msgr2=0x7fe6900f7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.458+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 shutdown_connections 2024-02-18T01:09:46.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:46.458+0000 7fe69560f700 1 -- 172.21.15.105:0/3903104221 wait complete. 2024-02-18T01:09:47.095 INFO:teuthology.orchestra.run.smithi105.stderr:Enabling the dashboard module... 2024-02-18T01:09:47.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:47 smithi105 bash[19734]: cluster 2024-02-18T01:09:45.618485+0000 mgr.y (mgr.14116) 15 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:09:47.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:47 smithi105 bash[19734]: audit 2024-02-18T01:09:46.023297+0000 mon.a (mon.0) 63 : audit [INF] from='mgr.14116 v1:172.21.15.105:0/1369308596' entity='mgr.y' 2024-02-18T01:09:47.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:47 smithi105 bash[19734]: audit 2024-02-18T01:09:46.454989+0000 mon.a (mon.0) 64 : audit [INF] from='client.? v1:172.21.15.105:0/3903104221' entity='client.admin' 2024-02-18T01:09:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.163+0000 7fea12920700 1 Processor -- start 2024-02-18T01:09:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.163+0000 7fea12920700 1 -- start start 2024-02-18T01:09:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.163+0000 7fea12920700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fea0c0fb0b0 con 0x7fea0c0fa580 2024-02-18T01:09:48.172 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.163+0000 7fea0bfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fea0c0fa580 0x7fea0c0fa9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48618/0 (socket says 172.21.15.105:48618) 2024-02-18T01:09:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.163+0000 7fea0bfff700 1 -- 172.21.15.105:0/871495734 learned_addr learned my addr 172.21.15.105:0/871495734 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2888630593 0 0) 0x7fea0c0fb0b0 con 0x7fea0c0fa580 2024-02-18T01:09:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f80033f0 con 0x7fea0c0fa580 2024-02-18T01:09:48.173 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2002615537 0 0) 0x7fe9f80033f0 con 0x7fea0c0fa580 2024-02-18T01:09:48.174 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c0ffb00 con 0x7fea0c0fa580 2024-02-18T01:09:48.174 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe9fc001590 con 0x7fea0c0fa580 2024-02-18T01:09:48.174 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fe9fc001c50 con 0x7fea0c0fa580 2024-02-18T01:09:48.174 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea0b7fe700 1 -- 172.21.15.105:0/871495734 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe9fc005670 con 0x7fea0c0fa580 2024-02-18T01:09:48.175 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea12920700 1 -- 172.21.15.105:0/871495734 >> v1:172.21.15.105:6789/0 conn(0x7fea0c0fa580 legacy=0x7fea0c0fa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:48.175 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea12920700 1 -- 172.21.15.105:0/871495734 shutdown_connections 2024-02-18T01:09:48.175 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea12920700 1 -- 172.21.15.105:0/871495734 >> 172.21.15.105:0/871495734 conn(0x7fea0c0f5dc0 msgr2=0x7fea0c0f81e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:48.175 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea12920700 1 -- 172.21.15.105:0/871495734 shutdown_connections 2024-02-18T01:09:48.176 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.167+0000 7fea12920700 1 -- 172.21.15.105:0/871495734 wait complete. 2024-02-18T01:09:48.176 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea12920700 1 Processor -- start 2024-02-18T01:09:48.176 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea12920700 1 -- start start 2024-02-18T01:09:48.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea12920700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fea0c06fbe0 con 0x7fea0c0fa580 2024-02-18T01:09:48.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea0bfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fea0c0fa580 0x7fea0c10ce40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48632/0 (socket says 172.21.15.105:48632) 2024-02-18T01:09:48.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea0bfff700 1 -- 172.21.15.105:0/2940885082 learned_addr learned my addr 172.21.15.105:0/2940885082 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:48.179 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060705633 0 0) 0x7fea0c06fbe0 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9f8005260 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3161486152 0 0) 0x7fe9f8005260 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fea0c06fbe0 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe9fc000ff0 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2721508296 0 0) 0x7fea0c06fbe0 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea0c06fd70 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0c10e560 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0c10e990 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fe9fc004d30 con 0x7fea0c0fa580 2024-02-18T01:09:48.180 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fe9fc005f70 con 0x7fea0c0fa580 2024-02-18T01:09:48.181 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.171+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 8) v1 ==== 38582+0+0 (unknown 93157125 0 0) 0x7fe9fc00f870 con 0x7fea0c0fa580 2024-02-18T01:09:48.181 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.175+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7fe9fc03eec0 con 0x7fea0c0fa580 2024-02-18T01:09:48.181 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.175+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec004f50 con 0x7fea0c0fa580 2024-02-18T01:09:48.193 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.187+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fe9fc010fa0 con 0x7fea0c0fa580 2024-02-18T01:09:48.607 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:48.599+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fe9ec005200 con 0x7fea0c0fa580 2024-02-18T01:09:49.033 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.027+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 2961485293 0 0) 0x7fe9fc03e430 con 0x7fea0c0fa580 2024-02-18T01:09:49.035 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.027+0000 7fea09ffb700 1 -- 172.21.15.105:0/2940885082 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (unknown 1498667528 0 0) 0x7fe9fc005950 con 0x7fea0c0fa580 2024-02-18T01:09:49.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 >> v1:172.21.15.105:6800/1813807489 conn(0x7fe9f8033930 legacy=0x7fe9f8035df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:49.041 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 >> v1:172.21.15.105:6789/0 conn(0x7fea0c0fa580 legacy=0x7fea0c10ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:49.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 shutdown_connections 2024-02-18T01:09:49.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 >> 172.21.15.105:0/2940885082 conn(0x7fea0c0f5dc0 msgr2=0x7fea0c0f69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:49.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 shutdown_connections 2024-02-18T01:09:49.042 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:49.035+0000 7fea12920700 1 -- 172.21.15.105:0/2940885082 wait complete. 2024-02-18T01:09:49.339 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[19734]: cluster 2024-02-18T01:09:47.618928+0000 mgr.y (mgr.14116) 16 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:09:49.339 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[19734]: audit 2024-02-18T01:09:48.603825+0000 mon.a (mon.0) 65 : audit [INF] from='client.? v1:172.21.15.105:0/2940885082' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-02-18T01:09:49.339 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[20046]: ignoring --setuser ceph since I am not root 2024-02-18T01:09:49.340 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[20046]: ignoring --setgroup ceph since I am not root 2024-02-18T01:09:49.340 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[20046]: debug 2024-02-18T01:09:49.227+0000 7f86f4561680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:09:49.742 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[20046]: debug 2024-02-18T01:09:49.331+0000 7f86f4561680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:09:50.039 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:49 smithi105 bash[20046]: debug 2024-02-18T01:09:49.775+0000 7f86f4561680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:09:50.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:50 smithi105 bash[19734]: audit 2024-02-18T01:09:49.031833+0000 mon.a (mon.0) 66 : audit [INF] from='client.? v1:172.21.15.105:0/2940885082' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-02-18T01:09:50.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:50 smithi105 bash[19734]: cluster 2024-02-18T01:09:49.032009+0000 mon.a (mon.0) 67 : cluster [DBG] mgrmap e9: y(active, since 23s) 2024-02-18T01:09:50.784 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:50 smithi105 bash[20046]: debug 2024-02-18T01:09:50.683+0000 7f86f4561680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:09:50.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed42b4700 1 Processor -- start 2024-02-18T01:09:50.883 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed42b4700 1 -- start start 2024-02-18T01:09:50.884 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed42b4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ecc072110 con 0x7f8ecc071cf0 2024-02-18T01:09:50.884 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed2050700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8ecc071cf0 0x7f8ecc070290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33410/0 (socket says 172.21.15.105:33410) 2024-02-18T01:09:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed2050700 1 -- 172.21.15.105:0/2325857383 learned_addr learned my addr 172.21.15.105:0/2325857383 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261052672 0 0) 0x7f8ecc072110 con 0x7f8ecc071cf0 2024-02-18T01:09:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.875+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb80033f0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.886 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2936970318 0 0) 0x7f8eb80033f0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0ff260 con 0x7f8ecc071cf0 2024-02-18T01:09:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f8ec8001590 con 0x7f8ecc071cf0 2024-02-18T01:09:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f8ec8001c50 con 0x7f8ecc071cf0 2024-02-18T01:09:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed104e700 1 -- 172.21.15.105:0/2325857383 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f8ec8005670 con 0x7f8ecc071cf0 2024-02-18T01:09:50.887 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- 172.21.15.105:0/2325857383 >> v1:172.21.15.105:6789/0 conn(0x7f8ecc071cf0 legacy=0x7f8ecc070290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:50.888 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- 172.21.15.105:0/2325857383 shutdown_connections 2024-02-18T01:09:50.888 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- 172.21.15.105:0/2325857383 >> 172.21.15.105:0/2325857383 conn(0x7f8ecc0f6e90 msgr2=0x7f8ecc0f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:50.888 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- 172.21.15.105:0/2325857383 shutdown_connections 2024-02-18T01:09:50.888 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- 172.21.15.105:0/2325857383 wait complete. 2024-02-18T01:09:50.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 Processor -- start 2024-02-18T01:09:50.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- start start 2024-02-18T01:09:50.894 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed42b4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8ecc110d90 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed2050700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f8ecc071cf0 0x7f8ecc110680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33412/0 (socket says 172.21.15.105:33412) 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.879+0000 7f8ed2050700 1 -- 172.21.15.105:0/1466596669 learned_addr learned my addr 172.21.15.105:0/1466596669 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1527199823 0 0) 0x7f8ecc110d90 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8eb8005260 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 617349217 0 0) 0x7f8eb8005260 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8ecc110d90 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f8ec8000ff0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1201841569 0 0) 0x7f8ecc110d90 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc110f20 con 0x7f8ecc071cf0 2024-02-18T01:09:50.895 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc1111d0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc114ec0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f8ec8004730 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f8ec8006090 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb0004f50 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 2961485293 0 0) 0x7f8ec800f9a0 con 0x7f8ecc071cf0 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ed184f700 1 -- 172.21.15.105:0/1466596669 >> v1:172.21.15.105:6800/1813807489 conn(0x7f8eb80338a0 legacy=0x7f8eb8035d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:50.896 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.883+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f8ec803ed20 con 0x7f8ecc071cf0 2024-02-18T01:09:50.898 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:50.891+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f8ec8011120 con 0x7f8ecc071cf0 2024-02-18T01:09:51.091 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.083+0000 7f8ed184f700 1 -- 172.21.15.105:0/1466596669 >> v1:172.21.15.105:6800/1813807489 conn(0x7f8eb80338a0 legacy=0x7f8eb8035d60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:51.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:50 smithi105 bash[20046]: debug 2024-02-18T01:09:50.779+0000 7f86f4561680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:09:51.242 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:51 smithi105 bash[20046]: debug 2024-02-18T01:09:51.023+0000 7f86f4561680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:09:51.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.295+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8eb0005a10 con 0x7f8ecc071cf0 2024-02-18T01:09:51.302 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.295+0000 7f8ec3779700 1 -- 172.21.15.105:0/1466596669 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (unknown 2005831594 0 1748205903) 0x7f8ec8013200 con 0x7f8ecc071cf0 2024-02-18T01:09:51.303 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "epoch": 9, 2024-02-18T01:09:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "available": true, 2024-02-18T01:09:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "active_name": "y", 2024-02-18T01:09:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "num_standby": 0 2024-02-18T01:09:51.304 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 >> v1:172.21.15.105:6800/1813807489 conn(0x7f8eb80338a0 legacy=0x7f8eb8035d60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T01:09:51.307 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 >> v1:172.21.15.105:6789/0 conn(0x7f8ecc071cf0 legacy=0x7f8ecc110680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 shutdown_connections 2024-02-18T01:09:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 >> 172.21.15.105:0/1466596669 conn(0x7f8ecc0f6e90 msgr2=0x7f8ecc0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:51.308 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 shutdown_connections 2024-02-18T01:09:51.309 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:51.299+0000 7f8ed42b4700 1 -- 172.21.15.105:0/1466596669 wait complete. 2024-02-18T01:09:51.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:51 smithi105 bash[19734]: audit 2024-02-18T01:09:51.299305+0000 mon.a (mon.0) 68 : audit [DBG] from='client.? v1:172.21.15.105:0/1466596669' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-02-18T01:09:51.884 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for the mgr to restart... 2024-02-18T01:09:51.884 INFO:teuthology.orchestra.run.smithi105.stderr:Waiting for mgr epoch 9... 2024-02-18T01:09:52.407 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:52 smithi105 bash[20046]: debug 2024-02-18T01:09:52.123+0000 7f86f4561680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:09:52.407 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:52 smithi105 bash[20046]: debug 2024-02-18T01:09:52.279+0000 7f86f4561680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:52.709 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:52 smithi105 bash[20046]: debug 2024-02-18T01:09:52.375+0000 7f86f4561680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:52.709 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:52 smithi105 bash[20046]: debug 2024-02-18T01:09:52.483+0000 7f86f4561680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:09:52.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:52 smithi105 bash[20046]: debug 2024-02-18T01:09:52.703+0000 7f86f4561680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:09:53.491 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:53 smithi105 bash[20046]: debug 2024-02-18T01:09:53.063+0000 7f86f4561680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:09:53.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:53 smithi105 bash[20046]: debug 2024-02-18T01:09:53.183+0000 7f86f4561680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:09:53.521 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 Processor -- start 2024-02-18T01:09:53.521 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- start start 2024-02-18T01:09:53.521 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c280720b0 con 0x7f0c28071c90 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2659c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0c28071c90 0x7f0c28070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33414/0 (socket says 172.21.15.105:33414) 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2659c700 1 -- 172.21.15.105:0/2782262322 learned_addr learned my addr 172.21.15.105:0/2782262322 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367582693 0 0) 0x7f0c280720b0 con 0x7f0c28071c90 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c100033f0 con 0x7f0c28071c90 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1685015867 0 0) 0x7f0c100033f0 con 0x7f0c28071c90 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c28070940 con 0x7f0c28071c90 2024-02-18T01:09:53.522 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0c18001590 con 0x7f0c28071c90 2024-02-18T01:09:53.523 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f0c18001d20 con 0x7f0c28071c90 2024-02-18T01:09:53.523 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2559a700 1 -- 172.21.15.105:0/2782262322 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0c180057a0 con 0x7f0c28071c90 2024-02-18T01:09:53.523 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2782262322 >> v1:172.21.15.105:6789/0 conn(0x7f0c28071c90 legacy=0x7f0c28070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:53.523 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2782262322 shutdown_connections 2024-02-18T01:09:53.524 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2782262322 >> 172.21.15.105:0/2782262322 conn(0x7f0c280f6e90 msgr2=0x7f0c280f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:53.525 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2782262322 shutdown_connections 2024-02-18T01:09:53.525 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.515+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2782262322 wait complete. 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2cba9700 1 Processor -- start 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2cba9700 1 -- start start 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2cba9700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0c28103ca0 con 0x7f0c28071c90 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2659c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f0c28071c90 0x7f0c28103590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33416/0 (socket says 172.21.15.105:33416) 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2659c700 1 -- 172.21.15.105:0/2482976709 learned_addr learned my addr 172.21.15.105:0/2482976709 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2117385488 0 0) 0x7f0c28103ca0 con 0x7f0c28071c90 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c10005210 con 0x7f0c28071c90 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2956350036 0 0) 0x7f0c10005210 con 0x7f0c28071c90 2024-02-18T01:09:53.526 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0c28103ca0 con 0x7f0c28071c90 2024-02-18T01:09:53.527 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0c180057a0 con 0x7f0c28071c90 2024-02-18T01:09:53.528 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3053696219 0 0) 0x7f0c28103ca0 con 0x7f0c28071c90 2024-02-18T01:09:53.528 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c28103e30 con 0x7f0c28071c90 2024-02-18T01:09:53.528 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c28104080 con 0x7f0c28071c90 2024-02-18T01:09:53.528 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c28107e40 con 0x7f0c28071c90 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f0c18004db0 con 0x7f0c28071c90 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f0c18006230 con 0x7f0c28071c90 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.519+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 9) v1 ==== 38595+0+0 (unknown 2961485293 0 0) 0x7f0c1800fb20 con 0x7f0c28071c90 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.523+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6800/1813807489 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c10037510 con 0x7f0c10033840 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.523+0000 7f0c25d9b700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/1813807489 conn(0x7f0c10033840 legacy=0x7f0c10035d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:53.529 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.523+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(2..2 src has 1..2) v4 ==== 883+0+0 (unknown 832073048 0 0) 0x7f0c1803efe0 con 0x7f0c28071c90 2024-02-18T01:09:53.729 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:53.723+0000 7f0c25d9b700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/1813807489 conn(0x7f0c10033840 legacy=0x7f0c10035d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:54.131 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:54.123+0000 7f0c25d9b700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/1813807489 conn(0x7f0c10033840 legacy=0x7f0c10035d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:54.491 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.059+0000 7f86f4561680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:09:54.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.159+0000 7f86f4561680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:09:54.492 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.275+0000 7f86f4561680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:09:54.779 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.487+0000 7f86f4561680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:09:54.930 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:54.923+0000 7f0c25d9b700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/1813807489 conn(0x7f0c10033840 legacy=0x7f0c10035d00 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/1813807489 2024-02-18T01:09:55.107 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.771+0000 7f86f4561680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:09:55.107 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:54 smithi105 bash[20046]: debug 2024-02-18T01:09:54.931+0000 7f86f4561680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:09:55.417 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[20046]: debug 2024-02-18T01:09:55.099+0000 7f86f4561680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:09:55.417 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[20046]: debug 2024-02-18T01:09:55.307+0000 7f86f4561680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:09:55.441 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:55.435+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mgrmap(e 10) v1 ==== 38454+0+0 (unknown 1426273129 0 0) 0x7f0c1803e570 con 0x7f0c28071c90 2024-02-18T01:09:55.442 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:55.435+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/1813807489 conn(0x7f0c10033840 legacy=0x7f0c10035d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T01:09:55.700 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[20046]: debug 2024-02-18T01:09:55.411+0000 7f86f4561680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:09:55.700 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[20046]: Warning: Permanently added '172.21.15.105' (ECDSA) to the list of known hosts. 2024-02-18T01:09:55.700 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: cluster 2024-02-18T01:09:55.434877+0000 mon.a (mon.0) 69 : cluster [INF] Active manager daemon y restarted 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: cluster 2024-02-18T01:09:55.435149+0000 mon.a (mon.0) 70 : cluster [INF] Activating manager daemon y 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: cluster 2024-02-18T01:09:55.439386+0000 mon.a (mon.0) 71 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: cluster 2024-02-18T01:09:55.439632+0000 mon.a (mon.0) 72 : cluster [DBG] mgrmap e10: y(active, starting, since 0.00458913s) 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: audit 2024-02-18T01:09:55.442841+0000 mon.a (mon.0) 73 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: audit 2024-02-18T01:09:55.444443+0000 mon.a (mon.0) 74 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: audit 2024-02-18T01:09:55.445961+0000 mon.a (mon.0) 75 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: audit 2024-02-18T01:09:55.446406+0000 mon.a (mon.0) 76 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: audit 2024-02-18T01:09:55.446775+0000 mon.a (mon.0) 77 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:09:55.701 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:55 smithi105 bash[19734]: cluster 2024-02-18T01:09:55.451362+0000 mon.a (mon.0) 78 : cluster [INF] Manager daemon y is now available 2024-02-18T01:09:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.439+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mgrmap(e 11) v1 ==== 38489+0+0 (unknown 2737546266 0 0) 0x7f0c180402a0 con 0x7f0c28071c90 2024-02-18T01:09:56.445 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.439+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6800/3087430119 -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c10037510 con 0x7f0c1803ec00 2024-02-18T01:09:56.448 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.443+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (unknown 0 0 4198994610) 0x7f0c10037510 con 0x7f0c1803ec00 2024-02-18T01:09:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.455+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 --> v1:172.21.15.105:6800/3087430119 -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0c08002440 con 0x7f0c1803ec00 2024-02-18T01:09:56.464 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c177fe700 1 -- 172.21.15.105:0/2482976709 <== mgr.14146 v1:172.21.15.105:6800/3087430119 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (unknown 0 0 3086460295) 0x7f0c08002440 con 0x7f0c1803ec00 2024-02-18T01:09:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: { 2024-02-18T01:09:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "mgrmap_epoch": 11, 2024-02-18T01:09:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: "initialized": true 2024-02-18T01:09:56.465 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: } 2024-02-18T01:09:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6800/3087430119 conn(0x7f0c1803ec00 legacy=0x7f0c1003a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 >> v1:172.21.15.105:6789/0 conn(0x7f0c28071c90 legacy=0x7f0c28103590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:56.466 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 shutdown_connections 2024-02-18T01:09:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 >> 172.21.15.105:0/2482976709 conn(0x7f0c280f6e90 msgr2=0x7f0c280ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 shutdown_connections 2024-02-18T01:09:56.467 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:56.459+0000 7f0c2cba9700 1 -- 172.21.15.105:0/2482976709 wait complete. 2024-02-18T01:09:56.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:56 smithi105 bash[19734]: audit 2024-02-18T01:09:55.590594+0000 mon.a (mon.0) 79 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:56.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:56 smithi105 bash[19734]: audit 2024-02-18T01:09:55.593599+0000 mon.a (mon.0) 80 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:09:56.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:56 smithi105 bash[19734]: audit 2024-02-18T01:09:55.643446+0000 mon.a (mon.0) 81 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:09:56.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:56 smithi105 bash[19734]: audit 2024-02-18T01:09:55.648127+0000 mon.a (mon.0) 82 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:09:56.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:56 smithi105 bash[19734]: cluster 2024-02-18T01:09:56.445444+0000 mon.a (mon.0) 83 : cluster [DBG] mgrmap e11: y(active, since 1.01039s) 2024-02-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:mgr epoch 9 is available 2024-02-18T01:09:57.197 INFO:teuthology.orchestra.run.smithi105.stderr:Generating a dashboard self-signed certificate... 2024-02-18T01:09:57.953 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:57 smithi105 bash[19734]: audit 2024-02-18T01:09:56.444295+0000 mgr.y (mgr.14146) 1 : audit [DBG] from='client.14150 v1:172.21.15.105:0/2482976709' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-02-18T01:09:57.953 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:57 smithi105 bash[19734]: audit 2024-02-18T01:09:56.461365+0000 mgr.y (mgr.14146) 2 : audit [DBG] from='client.14150 v1:172.21.15.105:0/2482976709' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-02-18T01:09:58.282 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 Processor -- start 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- start start 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f56680fe3c0 con 0x7f56680fd890 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566e0c9700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56680fd890 0x7f56680fdcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33490/0 (socket says 172.21.15.105:33490) 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566e0c9700 1 -- 172.21.15.105:0/2189588192 learned_addr learned my addr 172.21.15.105:0/2189588192 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406300540 0 0) 0x7f56680fe3c0 con 0x7f56680fd890 2024-02-18T01:09:58.283 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56500033f0 con 0x7f56680fd890 2024-02-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1613679449 0 0) 0x7f56500033f0 con 0x7f56680fd890 2024-02-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5668102e10 con 0x7f56680fd890 2024-02-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f5658001910 con 0x7f56680fd890 2024-02-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f56580020a0 con 0x7f56680fd890 2024-02-18T01:09:58.284 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f566d0c7700 1 -- 172.21.15.105:0/2189588192 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f5658005ac0 con 0x7f56680fd890 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- 172.21.15.105:0/2189588192 >> v1:172.21.15.105:6789/0 conn(0x7f56680fd890 legacy=0x7f56680fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- 172.21.15.105:0/2189588192 shutdown_connections 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- 172.21.15.105:0/2189588192 >> 172.21.15.105:0/2189588192 conn(0x7f5668070f30 msgr2=0x7f5668071340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- 172.21.15.105:0/2189588192 shutdown_connections 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.275+0000 7f567032d700 1 -- 172.21.15.105:0/2189588192 wait complete. 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f567032d700 1 Processor -- start 2024-02-18T01:09:58.286 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f567032d700 1 -- start start 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f567032d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f566810c960 con 0x7f56680fd890 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f566e0c9700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f56680fd890 0x7f566810c250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33506/0 (socket says 172.21.15.105:33506) 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f566e0c9700 1 -- 172.21.15.105:0/2102261373 learned_addr learned my addr 172.21.15.105:0/2102261373 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1950109164 0 0) 0x7f566810c960 con 0x7f56680fd890 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5650005210 con 0x7f56680fd890 2024-02-18T01:09:58.287 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 315732855 0 0) 0x7f5650005210 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f566810c960 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f5658001250 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 427332742 0 0) 0x7f566810c960 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f566810caf0 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f566810cda0 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5668110a90 con 0x7f56680fd890 2024-02-18T01:09:58.288 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f56580014f0 con 0x7f56680fd890 2024-02-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f56580064d0 con 0x7f56680fd890 2024-02-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.279+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 12) v1 ==== 38595+0+0 (unknown 2350480137 0 0) 0x7f565800fd20 con 0x7f56680fd890 2024-02-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.283+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f565803ffe0 con 0x7f56680fd890 2024-02-18T01:09:58.289 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.283+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5654004f50 con 0x7f56680fd890 2024-02-18T01:09:58.297 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.291+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5658011620 con 0x7f56680fd890 2024-02-18T01:09:58.634 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.627+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7f5654000bc0 con 0x7f56500338e0 2024-02-18T01:09:58.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.755+0000 7f56677fe700 1 -- 172.21.15.105:0/2102261373 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3317752739) 0x7f5654000bc0 con 0x7f56500338e0 2024-02-18T01:09:58.763 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: Self-signed certificate created 2024-02-18T01:09:58.766 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 >> v1:172.21.15.105:6800/3087430119 conn(0x7f56500338e0 legacy=0x7f5650035da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 >> v1:172.21.15.105:6789/0 conn(0x7f56680fd890 legacy=0x7f566810c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:09:58.767 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 shutdown_connections 2024-02-18T01:09:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 >> 172.21.15.105:0/2102261373 conn(0x7f5668070f30 msgr2=0x7f56680facd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:09:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 shutdown_connections 2024-02-18T01:09:58.768 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:09:58.759+0000 7f567032d700 1 -- 172.21.15.105:0/2102261373 wait complete. 2024-02-18T01:09:58.831 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:58 smithi105 bash[19734]: cluster 2024-02-18T01:09:57.498092+0000 mon.a (mon.0) 84 : cluster [DBG] mgrmap e12: y(active, since 2s) 2024-02-18T01:09:59.336 INFO:teuthology.orchestra.run.smithi105.stderr:Creating initial admin user... 2024-02-18T01:10:00.118 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:59 smithi105 bash[19734]: audit 2024-02-18T01:09:58.632723+0000 mgr.y (mgr.14146) 3 : audit [DBG] from='client.14156 v1:172.21.15.105:0/2102261373' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:10:00.118 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:59 smithi105 bash[19734]: audit 2024-02-18T01:09:58.755133+0000 mon.a (mon.0) 85 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:00.118 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:09:59 smithi105 bash[19734]: audit 2024-02-18T01:09:58.759563+0000 mon.a (mon.0) 86 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.415+0000 7f16e960a700 1 Processor -- start 2024-02-18T01:10:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.415+0000 7f16e960a700 1 -- start start 2024-02-18T01:10:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16e40ffa10 con 0x7f16e406fc30 2024-02-18T01:10:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e2ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f16e406fc30 0x7f16e40fe2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48624/0 (socket says 172.21.15.105:48624) 2024-02-18T01:10:00.424 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e2ffd700 1 -- 172.21.15.105:0/2255870459 learned_addr learned my addr 172.21.15.105:0/2255870459 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:00.425 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1697178835 0 0) 0x7f16e40ffa10 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d00033f0 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1504613329 0 0) 0x7f16d00033f0 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e4100bb0 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f16dc001590 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f16dc001c50 con 0x7f16e406fc30 2024-02-18T01:10:00.426 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e1ffb700 1 -- 172.21.15.105:0/2255870459 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f16dc005670 con 0x7f16e406fc30 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- 172.21.15.105:0/2255870459 >> v1:172.21.15.105:6789/0 conn(0x7f16e406fc30 legacy=0x7f16e40fe2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- 172.21.15.105:0/2255870459 shutdown_connections 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- 172.21.15.105:0/2255870459 >> 172.21.15.105:0/2255870459 conn(0x7f16e40f5d90 msgr2=0x7f16e40f81b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- 172.21.15.105:0/2255870459 shutdown_connections 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.419+0000 7f16e960a700 1 -- 172.21.15.105:0/2255870459 wait complete. 2024-02-18T01:10:00.427 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e960a700 1 Processor -- start 2024-02-18T01:10:00.428 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e960a700 1 -- start start 2024-02-18T01:10:00.428 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e960a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f16e410a790 con 0x7f16e406fc30 2024-02-18T01:10:00.428 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e2ffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f16e406fc30 0x7f16e410a080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48634/0 (socket says 172.21.15.105:48634) 2024-02-18T01:10:00.428 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e2ffd700 1 -- 172.21.15.105:0/3203491243 learned_addr learned my addr 172.21.15.105:0/3203491243 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:00.429 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1081633457 0 0) 0x7f16e410a790 con 0x7f16e406fc30 2024-02-18T01:10:00.429 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d0005260 con 0x7f16e406fc30 2024-02-18T01:10:00.430 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 996812910 0 0) 0x7f16d0005260 con 0x7f16e406fc30 2024-02-18T01:10:00.430 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f16e410a790 con 0x7f16e406fc30 2024-02-18T01:10:00.430 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f16dc0046b0 con 0x7f16e406fc30 2024-02-18T01:10:00.430 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4110078376 0 0) 0x7f16e410a790 con 0x7f16e406fc30 2024-02-18T01:10:00.430 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e410a920 con 0x7f16e406fc30 2024-02-18T01:10:00.431 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16e410abd0 con 0x7f16e406fc30 2024-02-18T01:10:00.431 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f16e410e8c0 con 0x7f16e406fc30 2024-02-18T01:10:00.432 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f16dc004cb0 con 0x7f16e406fc30 2024-02-18T01:10:00.432 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f16dc006080 con 0x7f16e406fc30 2024-02-18T01:10:00.432 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.423+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 12) v1 ==== 38595+0+0 (unknown 2350480137 0 0) 0x7f16dc00f8d0 con 0x7f16e406fc30 2024-02-18T01:10:00.432 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.427+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f16dc03fbb0 con 0x7f16e406fc30 2024-02-18T01:10:00.432 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.427+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c4004f50 con 0x7f16e406fc30 2024-02-18T01:10:00.438 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.431+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f16dc011040 con 0x7f16e406fc30 2024-02-18T01:10:00.795 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:00.787+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f16c4002430 con 0x7f16d0033a80 2024-02-18T01:10:01.049 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.043+0000 7f16dbfff700 1 -- 172.21.15.105:0/3203491243 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (unknown 0 0 2307213264) 0x7f16c4002430 con 0x7f16d0033a80 2024-02-18T01:10:01.050 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$XYcoMu.cmsXVEOpOcuq66.GG.xuvi.YCql2q9jmqxO1GRhduXlSr2", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1708218601, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.043+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 >> v1:172.21.15.105:6800/3087430119 conn(0x7f16d0033a80 legacy=0x7f16d0035f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.043+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 >> v1:172.21.15.105:6789/0 conn(0x7f16e406fc30 legacy=0x7f16e410a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.047+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 shutdown_connections 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.047+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 >> 172.21.15.105:0/3203491243 conn(0x7f16e40f5d90 msgr2=0x7f16e40fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.047+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 shutdown_connections 2024-02-18T01:10:01.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:01.047+0000 7f16e960a700 1 -- 172.21.15.105:0/3203491243 wait complete. 2024-02-18T01:10:01.650 INFO:teuthology.orchestra.run.smithi105.stderr:Fetching dashboard port number... 2024-02-18T01:10:02.336 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:02 smithi105 bash[19734]: audit 2024-02-18T01:10:00.793918+0000 mgr.y (mgr.14146) 4 : audit [DBG] from='client.14158 v1:172.21.15.105:0/3203491243' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:10:02.337 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:02 smithi105 bash[19734]: audit 2024-02-18T01:10:01.046628+0000 mon.a (mon.0) 87 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:02.337 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:02 smithi105 bash[19734]: audit 2024-02-18T01:10:01.134189+0000 mon.a (mon.0) 88 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.671+0000 7fb0985d4700 1 Processor -- start 2024-02-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.671+0000 7fb0985d4700 1 -- start start 2024-02-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.671+0000 7fb0985d4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb0900fa8b0 con 0x7fb0900f9d80 2024-02-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.671+0000 7fb096370700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb0900f9d80 0x7fb0900fa1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48640/0 (socket says 172.21.15.105:48640) 2024-02-18T01:10:02.679 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.671+0000 7fb096370700 1 -- 172.21.15.105:0/816367590 learned_addr learned my addr 172.21.15.105:0/816367590 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377197311 0 0) 0x7fb08c001110 con 0x7fb0900f9d80 2024-02-18T01:10:02.680 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c004500 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3666097026 0 0) 0x7fb07c004500 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0900ff1c0 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb08c001720 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb08c004770 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb09536e700 1 -- 172.21.15.105:0/816367590 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb08c005800 con 0x7fb0900f9d80 2024-02-18T01:10:02.681 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- 172.21.15.105:0/816367590 >> v1:172.21.15.105:6789/0 conn(0x7fb0900f9d80 legacy=0x7fb0900fa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- 172.21.15.105:0/816367590 shutdown_connections 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- 172.21.15.105:0/816367590 >> 172.21.15.105:0/816367590 conn(0x7fb0900f5520 msgr2=0x7fb0900f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- 172.21.15.105:0/816367590 shutdown_connections 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- 172.21.15.105:0/816367590 wait complete. 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 Processor -- start 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- start start 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0985d4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb09010a8e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb096370700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb0900f9d80 0x7fb09010a1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48652/0 (socket says 172.21.15.105:48652) 2024-02-18T01:10:02.687 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb096370700 1 -- 172.21.15.105:0/2921269567 learned_addr learned my addr 172.21.15.105:0/2921269567 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836566394 0 0) 0x7fb09010a8e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb07c006370 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 910805568 0 0) 0x7fb07c006370 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.675+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb09010a8e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb08c0048e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1351470122 0 0) 0x7fb09010a8e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.688 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09010aa70 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09010ad20 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb09010ea10 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb08c0019e0 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fb08c0061a0 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fb08c00fae0 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074004f50 con 0x7fb0900f9d80 2024-02-18T01:10:02.689 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.679+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7fb08c03ee20 con 0x7fb0900f9d80 2024-02-18T01:10:02.693 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:02.687+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb08c011170 con 0x7fb0900f9d80 2024-02-18T01:10:03.020 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fb074005780 con 0x7fb0900f9d80 2024-02-18T01:10:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0877fe700 1 -- 172.21.15.105:0/2921269567 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (unknown 3713421687 0 83753974) 0x7fb08c013250 con 0x7fb0900f9d80 2024-02-18T01:10:03.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 8443 2024-02-18T01:10:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb07c0339c0 legacy=0x7fb07c035e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:03.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 >> v1:172.21.15.105:6789/0 conn(0x7fb0900f9d80 legacy=0x7fb09010a1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 shutdown_connections 2024-02-18T01:10:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.015+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 >> 172.21.15.105:0/2921269567 conn(0x7fb0900f5520 msgr2=0x7fb0900f7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.019+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 shutdown_connections 2024-02-18T01:10:03.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:03.019+0000 7fb0985d4700 1 -- 172.21.15.105:0/2921269567 wait complete. 2024-02-18T01:10:03.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:03 smithi105 bash[19734]: cluster 2024-02-18T01:10:02.138683+0000 mon.a (mon.0) 89 : cluster [DBG] mgrmap e13: y(active, since 6s) 2024-02-18T01:10:03.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:03 smithi105 bash[19734]: audit 2024-02-18T01:10:03.018164+0000 mon.a (mon.0) 90 : audit [DBG] from='client.? v1:172.21.15.105:0/2921269567' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-02-18T01:10:03.649 INFO:teuthology.orchestra.run.smithi105.stderr:firewalld does not appear to be present 2024-02-18T01:10:03.649 INFO:teuthology.orchestra.run.smithi105.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr:Ceph Dashboard is now available at: 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: URL: https://smithi105.front.sepia.ceph.com:8443/ 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: User: admin 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: Password: koafbct3qc 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:03.652 INFO:teuthology.orchestra.run.smithi105.stderr:Enabling autotune for osd_memory_target 2024-02-18T01:10:04.711 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.703+0000 7f4a94fd4700 1 Processor -- start 2024-02-18T01:10:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.703+0000 7f4a94fd4700 1 -- start start 2024-02-18T01:10:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4a900720b0 con 0x7f4a90071c90 2024-02-18T01:10:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8e59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4a90071c90 0x7f4a90070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48656/0 (socket says 172.21.15.105:48656) 2024-02-18T01:10:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8e59c700 1 -- 172.21.15.105:0/4119210912 learned_addr learned my addr 172.21.15.105:0/4119210912 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:04.712 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096393101 0 0) 0x7f4a900720b0 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a7c0033f0 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911007850 0 0) 0x7f4a7c0033f0 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90070940 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a80001910 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4a800020a0 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8d59a700 1 -- 172.21.15.105:0/4119210912 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a80005ac0 con 0x7f4a90071c90 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- 172.21.15.105:0/4119210912 >> v1:172.21.15.105:6789/0 conn(0x7f4a90071c90 legacy=0x7f4a90070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- 172.21.15.105:0/4119210912 shutdown_connections 2024-02-18T01:10:04.714 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- 172.21.15.105:0/4119210912 >> 172.21.15.105:0/4119210912 conn(0x7f4a900f6e90 msgr2=0x7f4a900f92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- 172.21.15.105:0/4119210912 shutdown_connections 2024-02-18T01:10:04.715 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- 172.21.15.105:0/4119210912 wait complete. 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 Processor -- start 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- start start 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a94fd4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4a9010c930 con 0x7f4a90071c90 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8e59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4a90071c90 0x7f4a9010c220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48670/0 (socket says 172.21.15.105:48670) 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.707+0000 7f4a8e59c700 1 -- 172.21.15.105:0/132756166 learned_addr learned my addr 172.21.15.105:0/132756166 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1216278257 0 0) 0x7f4a9010c930 con 0x7f4a90071c90 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a7c005210 con 0x7f4a90071c90 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4278040800 0 0) 0x7f4a7c005210 con 0x7f4a90071c90 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4a9010c930 con 0x7f4a90071c90 2024-02-18T01:10:04.718 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a80004bf0 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1180579687 0 0) 0x7f4a9010c930 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9010cac0 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9010cd70 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a90110a60 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4a80004e50 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a80006640 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f4a800067c0 con 0x7f4a90071c90 2024-02-18T01:10:04.719 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f4a800400e0 con 0x7f4a90071c90 2024-02-18T01:10:04.720 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.711+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a70004f50 con 0x7f4a90071c90 2024-02-18T01:10:04.727 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:04.723+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4a800070c0 con 0x7f4a90071c90 2024-02-18T01:10:05.052 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f4a70005780 con 0x7f4a90071c90 2024-02-18T01:10:05.053 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a877fe700 1 -- 172.21.15.105:0/132756166 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (unknown 689008156 0 0) 0x7f4a80014dd0 con 0x7f4a90071c90 2024-02-18T01:10:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 >> v1:172.21.15.105:6800/3087430119 conn(0x7f4a7c033980 legacy=0x7f4a7c035e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 >> v1:172.21.15.105:6789/0 conn(0x7f4a90071c90 legacy=0x7f4a9010c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:05.055 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 shutdown_connections 2024-02-18T01:10:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.047+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 >> 172.21.15.105:0/132756166 conn(0x7f4a900f6e90 msgr2=0x7f4a900ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.051+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 shutdown_connections 2024-02-18T01:10:05.056 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:05.051+0000 7f4a94fd4700 1 -- 172.21.15.105:0/132756166 wait complete. 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4aa637b700 1 Processor -- start 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4aa637b700 1 -- start start 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4aa637b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4aa00fc640 con 0x7f4aa00fbb10 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9ffff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4aa00fbb10 0x7f4aa00fbf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48672/0 (socket says 172.21.15.105:48672) 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9ffff700 1 -- 172.21.15.105:0/632161702 learned_addr learned my addr 172.21.15.105:0/632161702 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761564581 0 0) 0x7f4aa00fc640 con 0x7f4aa00fbb10 2024-02-18T01:10:07.018 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a880033f0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 438948046 0 0) 0x7f4a880033f0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.019 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa00ffb00 con 0x7f4aa00fbb10 2024-02-18T01:10:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a90001590 con 0x7f4aa00fbb10 2024-02-18T01:10:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 <== mon.0 v1:172.21.15.105:6789/0 4 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4a90001d20 con 0x7f4aa00fbb10 2024-02-18T01:10:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.011+0000 7f4a9effd700 1 -- 172.21.15.105:0/632161702 <== mon.0 v1:172.21.15.105:6789/0 5 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a90005740 con 0x7f4aa00fbb10 2024-02-18T01:10:07.020 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/632161702 >> v1:172.21.15.105:6789/0 conn(0x7f4aa00fbb10 legacy=0x7f4aa00fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/632161702 shutdown_connections 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/632161702 >> 172.21.15.105:0/632161702 conn(0x7f4aa00f5520 msgr2=0x7f4aa00f7940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/632161702 shutdown_connections 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/632161702 wait complete. 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 Processor -- start 2024-02-18T01:10:07.021 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- start start 2024-02-18T01:10:07.022 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4aa00f7530 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9ffff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4aa00fbb10 0x7f4aa00f5a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48686/0 (socket says 172.21.15.105:48686) 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9ffff700 1 -- 172.21.15.105:0/2068064044 learned_addr learned my addr 172.21.15.105:0/2068064044 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 103781196 0 0) 0x7f4aa00f7530 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a88005210 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2124098979 0 0) 0x7f4a88005210 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4aa00f7530 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a90000fe0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2248593823 0 0) 0x7f4aa00f7530 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa00f76c0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.023 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.015+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa00f7970 con 0x7f4aa00fbb10 2024-02-18T01:10:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4a90004ad0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa00f6430 con 0x7f4aa00fbb10 2024-02-18T01:10:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f4a90006220 con 0x7f4aa00fbb10 2024-02-18T01:10:07.024 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f4a900074f0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.025 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f4a9003fc60 con 0x7f4aa00fbb10 2024-02-18T01:10:07.025 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.019+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa005a310 con 0x7f4aa00fbb10 2024-02-18T01:10:07.032 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.027+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4a90008230 con 0x7f4aa00fbb10 2024-02-18T01:10:07.554 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.547+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f4aa0046ce0 con 0x7f4aa00fbb10 2024-02-18T01:10:07.558 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.551+0000 7f4a9d7fa700 1 -- 172.21.15.105:0/2068064044 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v24)=0 set mgr/dashboard/cluster/status v24) v1 ==== 153+0+0 (unknown 2554848587 0 0) 0x7f4a90014950 con 0x7f4aa00fbb10 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 >> v1:172.21.15.105:6800/3087430119 conn(0x7f4a88033990 legacy=0x7f4a88035e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 >> v1:172.21.15.105:6789/0 conn(0x7f4aa00fbb10 legacy=0x7f4aa00f5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 shutdown_connections 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 >> 172.21.15.105:0/2068064044 conn(0x7f4aa00f5520 msgr2=0x7f4aa00f97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 shutdown_connections 2024-02-18T01:10:07.562 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: 2024-02-18T01:10:07.555+0000 7f4aa637b700 1 -- 172.21.15.105:0/2068064044 wait complete. 2024-02-18T01:10:07.563 INFO:teuthology.orchestra.run.smithi105.stderr:/usr/bin/ceph: set mgr/dashboard/cluster/status 2024-02-18T01:10:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-02-18T01:10:08.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.301 INFO:teuthology.orchestra.run.smithi105.stderr: sudo /usr/sbin/cephadm shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:10:08.301 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.301 INFO:teuthology.orchestra.run.smithi105.stderr:Or, if you are only running a single cluster on this host: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: sudo /usr/sbin/cephadm shell 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:Please consider enabling telemetry to help improve Ceph: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: ceph telemetry on 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr:For more information see: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.302 INFO:teuthology.orchestra.run.smithi105.stderr: https://docs.ceph.com/en/pacific/mgr/telemetry/ 2024-02-18T01:10:08.303 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:10:08.303 INFO:teuthology.orchestra.run.smithi105.stderr:Bootstrap complete. 2024-02-18T01:10:08.344 INFO:tasks.cephadm:Fetching config... 2024-02-18T01:10:08.344 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:10:08.344 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-02-18T01:10:08.397 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-02-18T01:10:08.397 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:10:08.397 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-02-18T01:10:08.444 INFO:tasks.cephadm:Fetching mon keyring... 2024-02-18T01:10:08.445 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:10:08.445 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/keyring of=/dev/stdout 2024-02-18T01:10:08.502 INFO:tasks.cephadm:Fetching pub ssh key... 2024-02-18T01:10:08.502 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:10:08.502 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-02-18T01:10:08.549 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-02-18T01:10:08.549 DEBUG:teuthology.orchestra.run.smithi105:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnpuppM8DyREG3Wu9GmGHavHGZklhdLksASVSW3iktRVra0xiGXTConE4UIwvCwTmFDDv30gNMHlRowWKW9LbtA+txqSifYM/Zw1m47KB8pzwOz+TictO9Khg1pYY5Wjra0fJc1NBh9RkXNMMK2g+o4f93g4q7fix9vjbrXEdFF3BXatOIEt1AoqbmH3lka7YTgHGA2u7tTGdOoij2CLbWsE3zU/7Ec+WfDOhKPI+Sqlq24ASlZtPr2J0X/RRPG/4GAc4NA1N5A3XQC0sg1bVsSFNBwEK3dXyJTl+vXAqeRqCCi2wrCdwVak3Qo22buB+FA41L7vplBGMLliovYnQLLZahHlHik4q4/4MCCnOCKFzMu47yMj9glrhE1xCYK6aaSU8fQQJjZ9Ad6uJlwG7ha7s81rfOuyELFojeQl/sYMG/KeXXym1hnRQls55wA91I/8j5yQKrE7wDdCWaX6YwPmvBGAsrUi7FWKuGtGYZLTh2c2E9mmJzmJghKxjtPkk= ceph-358e970a-cdfa-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T01:10:08.602 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:08 smithi105 bash[19734]: audit 2024-02-18T01:10:07.556785+0000 mon.a (mon.0) 91 : audit [INF] from='client.? v1:172.21.15.105:0/2068064044' entity='client.admin' 2024-02-18T01:10:08.614 INFO:teuthology.orchestra.run.smithi105.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnpuppM8DyREG3Wu9GmGHavHGZklhdLksASVSW3iktRVra0xiGXTConE4UIwvCwTmFDDv30gNMHlRowWKW9LbtA+txqSifYM/Zw1m47KB8pzwOz+TictO9Khg1pYY5Wjra0fJc1NBh9RkXNMMK2g+o4f93g4q7fix9vjbrXEdFF3BXatOIEt1AoqbmH3lka7YTgHGA2u7tTGdOoij2CLbWsE3zU/7Ec+WfDOhKPI+Sqlq24ASlZtPr2J0X/RRPG/4GAc4NA1N5A3XQC0sg1bVsSFNBwEK3dXyJTl+vXAqeRqCCi2wrCdwVak3Qo22buB+FA41L7vplBGMLliovYnQLLZahHlHik4q4/4MCCnOCKFzMu47yMj9glrhE1xCYK6aaSU8fQQJjZ9Ad6uJlwG7ha7s81rfOuyELFojeQl/sYMG/KeXXym1hnRQls55wA91I/8j5yQKrE7wDdCWaX6YwPmvBGAsrUi7FWKuGtGYZLTh2c2E9mmJzmJghKxjtPkk= ceph-358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:10:08.624 DEBUG:teuthology.orchestra.run.smithi155:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnpuppM8DyREG3Wu9GmGHavHGZklhdLksASVSW3iktRVra0xiGXTConE4UIwvCwTmFDDv30gNMHlRowWKW9LbtA+txqSifYM/Zw1m47KB8pzwOz+TictO9Khg1pYY5Wjra0fJc1NBh9RkXNMMK2g+o4f93g4q7fix9vjbrXEdFF3BXatOIEt1AoqbmH3lka7YTgHGA2u7tTGdOoij2CLbWsE3zU/7Ec+WfDOhKPI+Sqlq24ASlZtPr2J0X/RRPG/4GAc4NA1N5A3XQC0sg1bVsSFNBwEK3dXyJTl+vXAqeRqCCi2wrCdwVak3Qo22buB+FA41L7vplBGMLliovYnQLLZahHlHik4q4/4MCCnOCKFzMu47yMj9glrhE1xCYK6aaSU8fQQJjZ9Ad6uJlwG7ha7s81rfOuyELFojeQl/sYMG/KeXXym1hnRQls55wA91I/8j5yQKrE7wDdCWaX6YwPmvBGAsrUi7FWKuGtGYZLTh2c2E9mmJzmJghKxjtPkk= ceph-358e970a-cdfa-11ee-95ba-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-02-18T01:10:08.651 INFO:teuthology.orchestra.run.smithi155.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCnpuppM8DyREG3Wu9GmGHavHGZklhdLksASVSW3iktRVra0xiGXTConE4UIwvCwTmFDDv30gNMHlRowWKW9LbtA+txqSifYM/Zw1m47KB8pzwOz+TictO9Khg1pYY5Wjra0fJc1NBh9RkXNMMK2g+o4f93g4q7fix9vjbrXEdFF3BXatOIEt1AoqbmH3lka7YTgHGA2u7tTGdOoij2CLbWsE3zU/7Ec+WfDOhKPI+Sqlq24ASlZtPr2J0X/RRPG/4GAc4NA1N5A3XQC0sg1bVsSFNBwEK3dXyJTl+vXAqeRqCCi2wrCdwVak3Qo22buB+FA41L7vplBGMLliovYnQLLZahHlHik4q4/4MCCnOCKFzMu47yMj9glrhE1xCYK6aaSU8fQQJjZ9Ad6uJlwG7ha7s81rfOuyELFojeQl/sYMG/KeXXym1hnRQls55wA91I/8j5yQKrE7wDdCWaX6YwPmvBGAsrUi7FWKuGtGYZLTh2c2E9mmJzmJghKxjtPkk= ceph-358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:10:08.663 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-02-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.448+0000 7f9769f9b700 1 -- 172.21.15.105:0/12788560 >> v1:172.21.15.105:6789/0 conn(0x7f9764061110 legacy=0x7f97640fa130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.448+0000 7f9769f9b700 1 -- 172.21.15.105:0/12788560 shutdown_connections 2024-02-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.448+0000 7f9769f9b700 1 -- 172.21.15.105:0/12788560 >> 172.21.15.105:0/12788560 conn(0x7f97640f1b30 msgr2=0x7f97640f3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:12.454 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.448+0000 7f9769f9b700 1 -- 172.21.15.105:0/12788560 shutdown_connections 2024-02-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.448+0000 7f9769f9b700 1 -- 172.21.15.105:0/12788560 wait complete. 2024-02-18T01:10:12.455 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 Processor -- start 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 -- start start 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9764107190 con 0x7f9764061110 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f97637fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9764061110 0x7f9764106a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39738/0 (socket says 172.21.15.105:39738) 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f97637fe700 1 -- 172.21.15.105:0/2682019250 learned_addr learned my addr 172.21.15.105:0/2682019250 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158588977 0 0) 0x7f9764107190 con 0x7f9764061110 2024-02-18T01:10:12.456 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9750005260 con 0x7f9764061110 2024-02-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 81312016 0 0) 0x7f9750005260 con 0x7f9764061110 2024-02-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9764107190 con 0x7f9764061110 2024-02-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f9754002060 con 0x7f9764061110 2024-02-18T01:10:12.457 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1452438967 0 0) 0x7f9764107190 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9764107320 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97641075d0 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f976410b2c0 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f9754002850 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f97540038c0 con 0x7f9764061110 2024-02-18T01:10:12.458 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9744004f50 con 0x7f9764061110 2024-02-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.452+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f9754003b00 con 0x7f9764061110 2024-02-18T01:10:12.459 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.456+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f975403eca0 con 0x7f9764061110 2024-02-18T01:10:12.471 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.468+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f9754011670 con 0x7f9764061110 2024-02-18T01:10:12.659 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.656+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 --> v1:172.21.15.105:6789/0 -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f9744005780 con 0x7f9764061110 2024-02-18T01:10:12.663 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9760ff9700 1 -- 172.21.15.105:0/2682019250 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (unknown 3028693289 0 0) 0x7f97540131c0 con 0x7f9764061110 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 >> v1:172.21.15.105:6800/3087430119 conn(0x7f9750033960 legacy=0x7f9750035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 >> v1:172.21.15.105:6789/0 conn(0x7f9764061110 legacy=0x7f9764106a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 shutdown_connections 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 >> 172.21.15.105:0/2682019250 conn(0x7f97640f1b30 msgr2=0x7f97640f8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 shutdown_connections 2024-02-18T01:10:12.666 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:12.660+0000 7f9769f9b700 1 -- 172.21.15.105:0/2682019250 wait complete. 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:11.908682+0000 mon.a (mon.0) 92 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.287871+0000 mon.a (mon.0) 93 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi105", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.294315+0000 mon.a (mon.0) 94 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.296771+0000 mon.a (mon.0) 95 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.303363+0000 mon.a (mon.0) 96 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.662027+0000 mon.a (mon.0) 97 : audit [INF] from='client.? v1:172.21.15.105:0/2682019250' entity='client.admin' 2024-02-18T01:10:12.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.720169+0000 mon.a (mon.0) 98 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:10:12.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:12 smithi105 bash[19734]: audit 2024-02-18T01:10:12.726787+0000 mon.a (mon.0) 99 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:13.305 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-02-18T01:10:13.305 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1798597604 >> v1:172.21.15.105:6789/0 conn(0x7f84900f63d0 legacy=0x7f84900f67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1798597604 shutdown_connections 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1798597604 >> 172.21.15.105:0/1798597604 conn(0x7f84900f1b30 msgr2=0x7f84900f3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1798597604 shutdown_connections 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1798597604 wait complete. 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 Processor -- start 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- start start 2024-02-18T01:10:16.257 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f84900f3b40 con 0x7f84900f63d0 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f849504b700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f84900f63d0 0x7f84900f2030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39746/0 (socket says 172.21.15.105:39746) 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f849504b700 1 -- 172.21.15.105:0/1102908014 learned_addr learned my addr 172.21.15.105:0/1102908014 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1616174573 0 0) 0x7f84900f3b40 con 0x7f84900f63d0 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8478005260 con 0x7f84900f63d0 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2713523531 0 0) 0x7f8478005260 con 0x7f84900f63d0 2024-02-18T01:10:16.258 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f84900f3b40 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f84880023e0 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2566947342 0 0) 0x7f84900f3b40 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84900f3cd0 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84900f3f80 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.252+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f84900f2a40 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.256+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f8488002bd0 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.256+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f8488003c10 con 0x7f84900f63d0 2024-02-18T01:10:16.259 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.256+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847c004f50 con 0x7f84900f63d0 2024-02-18T01:10:16.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.256+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f8488003e10 con 0x7f84900f63d0 2024-02-18T01:10:16.260 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.256+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f848803f2d0 con 0x7f84900f63d0 2024-02-18T01:10:16.269 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.264+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f8488011350 con 0x7f84900f63d0 2024-02-18T01:10:16.460 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.456+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f847c000bc0 con 0x7f8478033960 2024-02-18T01:10:16.467 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f848e777700 1 -- 172.21.15.105:0/1102908014 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (unknown 0 0 0) 0x7f847c000bc0 con 0x7f8478033960 2024-02-18T01:10:16.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 >> v1:172.21.15.105:6800/3087430119 conn(0x7f8478033960 legacy=0x7f8478035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:16.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 >> v1:172.21.15.105:6789/0 conn(0x7f84900f63d0 legacy=0x7f84900f2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:16.469 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 shutdown_connections 2024-02-18T01:10:16.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 >> 172.21.15.105:0/1102908014 conn(0x7f84900f1b30 msgr2=0x7f84900fa1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:16.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 shutdown_connections 2024-02-18T01:10:16.470 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:16.464+0000 7f84972af700 1 -- 172.21.15.105:0/1102908014 wait complete. 2024-02-18T01:10:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:16 smithi105 bash[19734]: cluster 2024-02-18T01:10:15.447759+0000 mgr.y (mgr.14146) 5 : cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:16 smithi105 bash[19734]: audit 2024-02-18T01:10:16.465406+0000 mon.a (mon.0) 100 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:17.064 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi155 2024-02-18T01:10:17.064 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:10:17.064 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.conf 2024-02-18T01:10:17.071 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:10:17.071 DEBUG:teuthology.orchestra.run.smithi155:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:10:17.119 INFO:tasks.cephadm:Adding host smithi155 to orchestrator... 2024-02-18T01:10:17.119 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch host add smithi155 2024-02-18T01:10:17.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: audit 2024-02-18T01:10:16.500328+0000 mon.a (mon.0) 101 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: audit 2024-02-18T01:10:16.506289+0000 mon.a (mon.0) 102 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: audit 2024-02-18T01:10:16.507554+0000 mon.a (mon.0) 103 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: cephadm 2024-02-18T01:10:16.509475+0000 mgr.y (mgr.14146) 7 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: cephadm 2024-02-18T01:10:16.530305+0000 mgr.y (mgr.14146) 8 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: audit 2024-02-18T01:10:16.578544+0000 mon.a (mon.0) 104 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:17 smithi105 bash[19734]: audit 2024-02-18T01:10:16.586424+0000 mon.a (mon.0) 105 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:18 smithi105 bash[19734]: cluster 2024-02-18T01:10:17.448299+0000 mgr.y (mgr.14146) 9 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:20 smithi105 bash[19734]: cluster 2024-02-18T01:10:19.448730+0000 mgr.y (mgr.14146) 10 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:22 smithi105 bash[19734]: cluster 2024-02-18T01:10:21.449324+0000 mgr.y (mgr.14146) 11 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:24.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:24 smithi105 bash[19734]: cluster 2024-02-18T01:10:23.449737+0000 mgr.y (mgr.14146) 12 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:26 smithi105 bash[19734]: cluster 2024-02-18T01:10:25.450199+0000 mgr.y (mgr.14146) 13 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:28.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:28 smithi105 bash[19734]: cluster 2024-02-18T01:10:27.450680+0000 mgr.y (mgr.14146) 14 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:30 smithi105 bash[19734]: cluster 2024-02-18T01:10:29.451154+0000 mgr.y (mgr.14146) 15 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:32.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:32 smithi105 bash[19734]: cluster 2024-02-18T01:10:31.451617+0000 mgr.y (mgr.14146) 16 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:34 smithi105 bash[19734]: cluster 2024-02-18T01:10:33.452085+0000 mgr.y (mgr.14146) 17 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:36 smithi105 bash[19734]: cluster 2024-02-18T01:10:35.452567+0000 mgr.y (mgr.14146) 18 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:38 smithi105 bash[19734]: cluster 2024-02-18T01:10:37.453024+0000 mgr.y (mgr.14146) 19 : cluster [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:40 smithi105 bash[19734]: cluster 2024-02-18T01:10:39.453495+0000 mgr.y (mgr.14146) 20 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/360003353 >> v1:172.21.15.105:6789/0 conn(0x7fc1c00f96f0 legacy=0x7fc1c00f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/360003353 shutdown_connections 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/360003353 >> 172.21.15.155:0/360003353 conn(0x7fc1c00622c0 msgr2=0x7fc1c00f7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/360003353 shutdown_connections 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/360003353 wait complete. 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 Processor -- start 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- start start 2024-02-18T01:10:42.187 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc1c0109360 con 0x7fc1c00f96f0 2024-02-18T01:10:42.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c4956700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fc1c00f96f0 0x7fc1c0108c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.155:45430/0 (socket says 172.21.15.155:45430) 2024-02-18T01:10:42.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c4956700 1 -- 172.21.15.155:0/4201300016 learned_addr learned my addr 172.21.15.155:0/4201300016 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:10:42.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 464537639 0 0) 0x7fc1c0109360 con 0x7fc1c00f96f0 2024-02-18T01:10:42.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ac005260 con 0x7fc1c00f96f0 2024-02-18T01:10:42.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4048264789 0 0) 0x7fc1ac005260 con 0x7fc1c00f96f0 2024-02-18T01:10:42.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc1c0109360 con 0x7fc1c00f96f0 2024-02-18T01:10:42.189 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fc1b00023e0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4209745556 0 0) 0x7fc1c0109360 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c01094f0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c01097a0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.184+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c010d490 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.188+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fc1b0002bd0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.190 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.188+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fc1b0003c10 con 0x7fc1c00f96f0 2024-02-18T01:10:42.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.188+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a0004f50 con 0x7fc1c00f96f0 2024-02-18T01:10:42.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.188+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fc1b0003530 con 0x7fc1c00f96f0 2024-02-18T01:10:42.191 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.188+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7fc1b003f0a0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.200 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.196+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc1b00042e0 con 0x7fc1c00f96f0 2024-02-18T01:10:42.387 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:42.384+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1a0000bc0 con 0x7fc1ac033960 2024-02-18T01:10:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:42 smithi105 bash[19734]: cluster 2024-02-18T01:10:41.453988+0000 mgr.y (mgr.14146) 21 : cluster [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:42.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:10:42 smithi105 bash[20046]: Warning: Permanently added 'smithi155,172.21.15.155' (ECDSA) to the list of known hosts. 2024-02-18T01:10:43.244 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.240+0000 7fc1bdffb700 1 -- 172.21.15.155:0/4201300016 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (unknown 0 0 2103545083) 0x7fc1a0000bc0 con 0x7fc1ac033960 2024-02-18T01:10:43.244 INFO:teuthology.orchestra.run.smithi155.stdout:Added host 'smithi155' with addr '172.21.15.155' 2024-02-18T01:10:43.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 >> v1:172.21.15.105:6800/3087430119 conn(0x7fc1ac033960 legacy=0x7fc1ac035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:43.246 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 >> v1:172.21.15.105:6789/0 conn(0x7fc1c00f96f0 legacy=0x7fc1c0108c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:43.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 shutdown_connections 2024-02-18T01:10:43.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 >> 172.21.15.155:0/4201300016 conn(0x7fc1c00622c0 msgr2=0x7fc1c00fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:43.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 shutdown_connections 2024-02-18T01:10:43.247 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:43.244+0000 7fc1c6bba700 1 -- 172.21.15.155:0/4201300016 wait complete. 2024-02-18T01:10:43.615 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:10:43 smithi105 bash[20046]: Warning: Permanently added '172.21.15.155' (ECDSA) to the list of known hosts. 2024-02-18T01:10:43.972 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch host ls --format=json 2024-02-18T01:10:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:43 smithi105 bash[19734]: audit 2024-02-18T01:10:42.388402+0000 mgr.y (mgr.14146) 22 : audit [DBG] from='client.14170 v1:172.21.15.155:0/4201300016' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi155", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:10:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:43 smithi105 bash[19734]: audit 2024-02-18T01:10:43.241659+0000 mon.a (mon.0) 106 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:43 smithi105 bash[19734]: audit 2024-02-18T01:10:43.275548+0000 mon.a (mon.0) 107 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:10:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:44 smithi105 bash[19734]: cephadm 2024-02-18T01:10:43.242209+0000 mgr.y (mgr.14146) 23 : cephadm [INF] Added host smithi155 2024-02-18T01:10:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:44 smithi105 bash[19734]: cluster 2024-02-18T01:10:43.454587+0000 mgr.y (mgr.14146) 24 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:44 smithi105 bash[19734]: audit 2024-02-18T01:10:43.859475+0000 mon.a (mon.0) 108 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:46 smithi105 bash[19734]: audit 2024-02-18T01:10:45.164633+0000 mon.a (mon.0) 109 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:46 smithi105 bash[19734]: cluster 2024-02-18T01:10:45.455076+0000 mgr.y (mgr.14146) 25 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:46.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- 172.21.15.155:0/314201750 >> v1:172.21.15.105:6789/0 conn(0x7fee780f74d0 legacy=0x7fee780f78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:46.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- 172.21.15.155:0/314201750 shutdown_connections 2024-02-18T01:10:46.670 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- 172.21.15.155:0/314201750 >> 172.21.15.155:0/314201750 conn(0x7fee780f2c30 msgr2=0x7fee780f5050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:46.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- 172.21.15.155:0/314201750 shutdown_connections 2024-02-18T01:10:46.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- 172.21.15.155:0/314201750 wait complete. 2024-02-18T01:10:46.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 Processor -- start 2024-02-18T01:10:46.671 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- start start 2024-02-18T01:10:46.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee80718700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fee781070e0 con 0x7fee780f74d0 2024-02-18T01:10:46.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee7e4b4700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fee780f74d0 0x7fee781069d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.155:57006/0 (socket says 172.21.15.155:57006) 2024-02-18T01:10:46.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee7e4b4700 1 -- 172.21.15.155:0/3790237074 learned_addr learned my addr 172.21.15.155:0/3790237074 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:10:46.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874136303 0 0) 0x7fee781070e0 con 0x7fee780f74d0 2024-02-18T01:10:46.672 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.668+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee60005260 con 0x7fee780f74d0 2024-02-18T01:10:46.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 785896162 0 0) 0x7fee60005260 con 0x7fee780f74d0 2024-02-18T01:10:46.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fee781070e0 con 0x7fee780f74d0 2024-02-18T01:10:46.673 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee6c001f00 con 0x7fee780f74d0 2024-02-18T01:10:46.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1616104393 0 0) 0x7fee781070e0 con 0x7fee780f74d0 2024-02-18T01:10:46.674 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee78107270 con 0x7fee780f74d0 2024-02-18T01:10:46.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee78107520 con 0x7fee780f74d0 2024-02-18T01:10:46.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7810b210 con 0x7fee780f74d0 2024-02-18T01:10:46.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fee6c0026f0 con 0x7fee780f74d0 2024-02-18T01:10:46.675 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fee6c0039c0 con 0x7fee780f74d0 2024-02-18T01:10:46.676 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee64004f50 con 0x7fee780f74d0 2024-02-18T01:10:46.676 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fee6c003ba0 con 0x7fee780f74d0 2024-02-18T01:10:46.676 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.672+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7fee6c03ef20 con 0x7fee780f74d0 2024-02-18T01:10:46.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.684+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fee6c0111a0 con 0x7fee780f74d0 2024-02-18T01:10:46.878 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.876+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fee64000bc0 con 0x7fee60033960 2024-02-18T01:10:46.900 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee777fe700 1 -- 172.21.15.155:0/3790237074 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (unknown 0 0 3419246945) 0x7fee64000bc0 con 0x7fee60033960 2024-02-18T01:10:46.900 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-18T01:10:46.900 INFO:teuthology.orchestra.run.smithi155.stdout:[{"addr": "172.21.15.105", "hostname": "smithi105", "labels": [], "status": ""}, {"addr": "172.21.15.155", "hostname": "smithi155", "labels": [], "status": ""}] 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 >> v1:172.21.15.105:6800/3087430119 conn(0x7fee60033960 legacy=0x7fee60035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 >> v1:172.21.15.105:6789/0 conn(0x7fee780f74d0 legacy=0x7fee781069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 shutdown_connections 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 >> 172.21.15.155:0/3790237074 conn(0x7fee780f2c30 msgr2=0x7fee780fb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 shutdown_connections 2024-02-18T01:10:46.901 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:46.880+0000 7fee80718700 1 -- 172.21.15.155:0/3790237074 wait complete. 2024-02-18T01:10:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:47 smithi105 bash[19734]: audit 2024-02-18T01:10:46.878982+0000 mgr.y (mgr.14146) 26 : audit [DBG] from='client.14172 v1:172.21.15.155:0/3790237074' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:10:47.704 INFO:tasks.cephadm:Setting crush tunables to default 2024-02-18T01:10:47.704 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd crush tunables default 2024-02-18T01:10:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:48 smithi105 bash[19734]: cluster 2024-02-18T01:10:47.455491+0000 mgr.y (mgr.14146) 27 : cluster [DBG] pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:50.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 -- 172.21.15.105:0/822467083 >> v1:172.21.15.105:6789/0 conn(0x7f3f640fd4f0 legacy=0x7f3f640fd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:50.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 -- 172.21.15.105:0/822467083 shutdown_connections 2024-02-18T01:10:50.425 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 -- 172.21.15.105:0/822467083 >> 172.21.15.105:0/822467083 conn(0x7f3f640f1770 msgr2=0x7f3f640f3b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:50.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 -- 172.21.15.105:0/822467083 shutdown_connections 2024-02-18T01:10:50.426 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 -- 172.21.15.105:0/822467083 wait complete. 2024-02-18T01:10:50.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.419+0000 7f3f68dcf700 1 Processor -- start 2024-02-18T01:10:50.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f68dcf700 1 -- start start 2024-02-18T01:10:50.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f68dcf700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3f6410d160 con 0x7f3f640fd4f0 2024-02-18T01:10:50.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f6259c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f3f640fd4f0 0x7f3f6410ca50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:45034/0 (socket says 172.21.15.105:45034) 2024-02-18T01:10:50.430 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f6259c700 1 -- 172.21.15.105:0/1575852478 learned_addr learned my addr 172.21.15.105:0/1575852478 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:10:50.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3030963838 0 0) 0x7f3f6410d160 con 0x7f3f640fd4f0 2024-02-18T01:10:50.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f4c005210 con 0x7f3f640fd4f0 2024-02-18T01:10:50.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 121191209 0 0) 0x7f3f4c005210 con 0x7f3f640fd4f0 2024-02-18T01:10:50.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3f6410d160 con 0x7f3f640fd4f0 2024-02-18T01:10:50.431 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3f58001430 con 0x7f3f640fd4f0 2024-02-18T01:10:50.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1372500111 0 0) 0x7f3f6410d160 con 0x7f3f640fd4f0 2024-02-18T01:10:50.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6410d2f0 con 0x7f3f640fd4f0 2024-02-18T01:10:50.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f6410d5a0 con 0x7f3f640fd4f0 2024-02-18T01:10:50.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f6410faa0 con 0x7f3f640fd4f0 2024-02-18T01:10:50.432 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f3f580011a0 con 0x7f3f640fd4f0 2024-02-18T01:10:50.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f3f58003770 con 0x7f3f640fd4f0 2024-02-18T01:10:50.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f44004f50 con 0x7f3f640fd4f0 2024-02-18T01:10:50.433 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.423+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f3f58003970 con 0x7f3f640fd4f0 2024-02-18T01:10:50.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.427+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(3..3 src has 1..3) v4 ==== 1012+0+0 (unknown 3004112264 0 0) 0x7f3f5803ed20 con 0x7f3f640fd4f0 2024-02-18T01:10:50.438 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.431+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f3f58010fc0 con 0x7f3f640fd4f0 2024-02-18T01:10:50.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:50.619+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f3f44005200 con 0x7f3f640fd4f0 2024-02-18T01:10:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:50 smithi105 bash[19734]: cluster 2024-02-18T01:10:49.455969+0000 mgr.y (mgr.14146) 28 : cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:51.520 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.515+0000 7f3f537fe700 1 -- 172.21.15.105:0/1575852478 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (unknown 3126668360 0 0) 0x7f3f58013080 con 0x7f3f640fd4f0 2024-02-18T01:10:51.524 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 >> v1:172.21.15.105:6800/3087430119 conn(0x7f3f4c033960 legacy=0x7f3f4c035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 >> v1:172.21.15.105:6789/0 conn(0x7f3f640fd4f0 legacy=0x7f3f6410ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 shutdown_connections 2024-02-18T01:10:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 >> 172.21.15.105:0/1575852478 conn(0x7f3f640f1770 msgr2=0x7f3f640ff8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 shutdown_connections 2024-02-18T01:10:51.525 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:10:51.519+0000 7f3f68dcf700 1 -- 172.21.15.105:0/1575852478 wait complete. 2024-02-18T01:10:51.526 INFO:teuthology.orchestra.run.smithi105.stderr:adjusted tunables profile to default 2024-02-18T01:10:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:50.625398+0000 mon.a (mon.0) 110 : audit [INF] from='client.? v1:172.21.15.105:0/1575852478' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-02-18T01:10:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:50.833993+0000 mon.a (mon.0) 111 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.136336+0000 mon.a (mon.0) 112 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.137910+0000 mon.a (mon.0) 113 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi155", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:10:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.140159+0000 mon.a (mon.0) 114 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:10:51.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.142205+0000 mon.a (mon.0) 115 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:10:51.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.221023+0000 mon.a (mon.0) 116 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:51.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:51 smithi105 bash[19734]: audit 2024-02-18T01:10:51.227141+0000 mon.a (mon.0) 117 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:52.188 INFO:tasks.cephadm:Adding mon.a on smithi105 2024-02-18T01:10:52.188 INFO:tasks.cephadm:Adding mon.c on smithi105 2024-02-18T01:10:52.189 INFO:tasks.cephadm:Adding mon.b on smithi155 2024-02-18T01:10:52.189 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply mon '3;smithi105:[v1:172.21.15.105:6789]=a;smithi105:[v1:172.21.15.105:6790]=c;smithi155:[v1:172.21.15.155:6789]=b' 2024-02-18T01:10:52.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:52 smithi105 bash[19734]: cephadm 2024-02-18T01:10:51.145068+0000 mgr.y (mgr.14146) 29 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:10:52.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:52 smithi105 bash[19734]: cephadm 2024-02-18T01:10:51.190852+0000 mgr.y (mgr.14146) 30 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:10:52.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:52 smithi105 bash[19734]: cluster 2024-02-18T01:10:51.456419+0000 mgr.y (mgr.14146) 31 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:52.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:52 smithi105 bash[19734]: audit 2024-02-18T01:10:51.519313+0000 mon.a (mon.0) 118 : audit [INF] from='client.? v1:172.21.15.105:0/1575852478' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-02-18T01:10:52.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:52 smithi105 bash[19734]: cluster 2024-02-18T01:10:51.519429+0000 mon.a (mon.0) 119 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:10:54.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.976+0000 7fcc05792700 1 -- 172.21.15.155:0/790784337 >> v1:172.21.15.105:6789/0 conn(0x7fcc000fd320 legacy=0x7fcc000fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:54.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.976+0000 7fcc05792700 1 -- 172.21.15.155:0/790784337 shutdown_connections 2024-02-18T01:10:54.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.976+0000 7fcc05792700 1 -- 172.21.15.155:0/790784337 >> 172.21.15.155:0/790784337 conn(0x7fcc000f2f50 msgr2=0x7fcc000f5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:54.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.976+0000 7fcc05792700 1 -- 172.21.15.155:0/790784337 shutdown_connections 2024-02-18T01:10:54.981 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.976+0000 7fcc05792700 1 -- 172.21.15.155:0/790784337 wait complete. 2024-02-18T01:10:54.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 Processor -- start 2024-02-18T01:10:54.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 -- start start 2024-02-18T01:10:54.982 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcc0006cee0 con 0x7fcc000fd320 2024-02-18T01:10:54.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfffff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcc000fd320 0x7fcc0006b0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.155:45196/0 (socket says 172.21.15.155:45196) 2024-02-18T01:10:54.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfffff700 1 -- 172.21.15.155:0/980183444 learned_addr learned my addr 172.21.15.155:0/980183444 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:10:54.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669545597 0 0) 0x7fcc0006cee0 con 0x7fcc000fd320 2024-02-18T01:10:54.983 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbe0005210 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2672193833 0 0) 0x7fcbe0005210 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcc0006cee0 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcbf80017b0 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 730533450 0 0) 0x7fcc0006cee0 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0006b7e0 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0006ba90 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc00111cb0 con 0x7fcc000fd320 2024-02-18T01:10:54.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fcbf8001480 con 0x7fcc000fd320 2024-02-18T01:10:54.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7fcbf8003b90 con 0x7fcc000fd320 2024-02-18T01:10:54.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.980+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe4004f50 con 0x7fcc000fd320 2024-02-18T01:10:54.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.984+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fcbf80034d0 con 0x7fcc000fd320 2024-02-18T01:10:54.985 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.984+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7fcbf803efc0 con 0x7fcc000fd320 2024-02-18T01:10:54.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:54 smithi105 bash[19734]: cluster 2024-02-18T01:10:53.456838+0000 mgr.y (mgr.14146) 32 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:54.998 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:54.996+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fcbf80042b0 con 0x7fcc000fd320 2024-02-18T01:10:55.185 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.184+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:[v1:172.21.15.105:6789]=a;smithi105:[v1:172.21.15.105:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fcbe4000ca0 con 0x7fcbe0033920 2024-02-18T01:10:55.200 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.196+0000 7fcbfd7fa700 1 -- 172.21.15.155:0/980183444 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 3265049985) 0x7fcbe4000ca0 con 0x7fcbe0033920 2024-02-18T01:10:55.200 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mon update... 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 >> v1:172.21.15.105:6800/3087430119 conn(0x7fcbe0033920 legacy=0x7fcbe0035de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 >> v1:172.21.15.105:6789/0 conn(0x7fcc000fd320 legacy=0x7fcc0006b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 shutdown_connections 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 >> 172.21.15.155:0/980183444 conn(0x7fcc000f2f50 msgr2=0x7fcc000ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 shutdown_connections 2024-02-18T01:10:55.203 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:55.200+0000 7fcc05792700 1 -- 172.21.15.155:0/980183444 wait complete. 2024-02-18T01:10:55.939 DEBUG:teuthology.orchestra.run.smithi105:mon.c> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.c.service 2024-02-18T01:10:55.941 DEBUG:teuthology.orchestra.run.smithi155:mon.b> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.b.service 2024-02-18T01:10:55.944 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T01:10:55.944 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T01:10:56.122 INFO:journalctl@ceph.mon.c.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.186929+0000 mgr.y (mgr.14146) 33 : audit [DBG] from='client.14176 v1:172.21.15.155:0/980183444' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi105:[v1:172.21.15.105:6789]=a;smithi105:[v1:172.21.15.105:6790]=c;smithi155:[v1:172.21.15.155:6789]=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: cephadm 2024-02-18T01:10:55.192925+0000 mgr.y (mgr.14146) 34 : cephadm [INF] Saving service mon spec with placement smithi105:[v1:172.21.15.105:6789]=a;smithi105:[v1:172.21.15.105:6790]=c;smithi155:[v1:172.21.15.155:6789]=b;count:3 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.198362+0000 mon.a (mon.0) 120 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.202308+0000 mon.a (mon.0) 121 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.205697+0000 mon.a (mon.0) 122 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:10:56.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.207512+0000 mon.a (mon.0) 123 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:10:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.217511+0000 mon.a (mon.0) 124 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:10:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.221100+0000 mon.a (mon.0) 125 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:10:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: audit 2024-02-18T01:10:55.222701+0000 mon.a (mon.0) 126 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:10:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: cephadm 2024-02-18T01:10:55.224116+0000 mgr.y (mgr.14146) 35 : cephadm [INF] Deploying daemon mon.b on smithi155 2024-02-18T01:10:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:56 smithi105 bash[19734]: cluster 2024-02-18T01:10:55.457299+0000 mgr.y (mgr.14146) 36 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:56.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:10:58.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:10:58 smithi105 bash[19734]: cluster 2024-02-18T01:10:57.457722+0000 mgr.y (mgr.14146) 37 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:10:59.518 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- 172.21.15.155:0/1087153217 >> v1:172.21.15.105:6789/0 conn(0x7f27b40fc420 legacy=0x7f27b40fc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:59.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- 172.21.15.155:0/1087153217 shutdown_connections 2024-02-18T01:10:59.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- 172.21.15.155:0/1087153217 >> 172.21.15.155:0/1087153217 conn(0x7f27b406c300 msgr2=0x7f27b406c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:59.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- 172.21.15.155:0/1087153217 shutdown_connections 2024-02-18T01:10:59.519 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- 172.21.15.155:0/1087153217 wait complete. 2024-02-18T01:10:59.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 Processor -- start 2024-02-18T01:10:59.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- start start 2024-02-18T01:10:59.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27bb405700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f27b410c070 con 0x7f27b40fc420 2024-02-18T01:10:59.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27b91a1700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f27b40fc420 0x7f27b410b960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.155:45210/0 (socket says 172.21.15.155:45210) 2024-02-18T01:10:59.520 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27b91a1700 1 -- 172.21.15.155:0/3008638211 learned_addr learned my addr 172.21.15.155:0/3008638211 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278480171 0 0) 0x7f27b410c070 con 0x7f27b40fc420 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27a0005260 con 0x7f27b40fc420 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3496290317 0 0) 0x7f27a0005260 con 0x7f27b40fc420 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f27b410c070 con 0x7f27b40fc420 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.516+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f27b0001490 con 0x7f27b40fc420 2024-02-18T01:10:59.521 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1239964204 0 0) 0x7f27b410c070 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b410c200 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b410c4b0 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b410e9f0 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f27b00023a0 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 170+0+0 (unknown 1369293703 0 0) 0x7f27b0003790 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2798004f50 con 0x7f27b40fc420 2024-02-18T01:10:59.522 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f27b0001f00 con 0x7f27b40fc420 2024-02-18T01:10:59.523 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.520+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7f27b003ef70 con 0x7f27b40fc420 2024-02-18T01:10:59.532 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.528+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f27b0013ed0 con 0x7f27b40fc420 2024-02-18T01:10:59.784 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.780+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f2798005200 con 0x7f27b40fc420 2024-02-18T01:10:59.785 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.780+0000 7f27a67fc700 1 -- 172.21.15.155:0/3008638211 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+675 (unknown 2237029548 0 2845706462) 0x7f27b0013cf0 con 0x7f27b40fc420 2024-02-18T01:10:59.785 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-18T01:10:59.785 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":1,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","modified":"2024-02-18T01:08:53.400007Z","created":"2024-02-18T01:08:53.400007Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 >> v1:172.21.15.105:6800/3087430119 conn(0x7f27a0033960 legacy=0x7f27a0035e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 >> v1:172.21.15.105:6789/0 conn(0x7f27b40fc420 legacy=0x7f27b410b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 shutdown_connections 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 >> 172.21.15.155:0/3008638211 conn(0x7f27b406c300 msgr2=0x7f27b40f5150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 shutdown_connections 2024-02-18T01:10:59.787 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:10:59.784+0000 7f27bb405700 1 -- 172.21.15.155:0/3008638211 wait complete. 2024-02-18T01:10:59.788 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 1 2024-02-18T01:11:00.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:00 smithi155 systemd[1]: Started Ceph mon.b for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:11:00.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:00 smithi105 bash[19734]: cluster 2024-02-18T01:10:59.458106+0000 mgr.y (mgr.14146) 38 : cluster [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:00 smithi105 bash[19734]: audit 2024-02-18T01:10:59.783929+0000 mon.a (mon.0) 127 : audit [DBG] from='client.? v1:172.21.15.155:0/3008638211' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T01:11:01.419 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T01:11:01.420 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T01:11:01.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.620+0000 7f68b8543880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.620+0000 7f68b8543880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 7 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.620+0000 7f68b8543880 0 pidfile_write: ignore empty --pid-file 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 0 load: jerasure load: lrc load: isa 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T01:11:01.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T01:11:01.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: DB SUMMARY 2024-02-18T01:11:01.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T01:11:01.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T01:11:01.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-18T01:11:01.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000003.log size: 476 ; 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.env: 0x562852a10080 2024-02-18T01:11:01.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.fs: Posix File System 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.info_log: 0x562853ce3980 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.statistics: (nil) 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.use_fsync: 0 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T01:11:01.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T01:11:01.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T01:11:01.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T01:11:01.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T01:11:01.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T01:11:01.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T01:11:01.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T01:11:01.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T01:11:01.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T01:11:01.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T01:11:01.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.db_log_dir: 2024-02-18T01:11:01.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T01:11:01.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T01:11:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T01:11:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T01:11:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T01:11:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T01:11:01.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.write_buffer_manager: 0x5628549e23c0 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T01:11:01.756 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T01:11:01.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T01:11:01.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T01:11:01.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T01:11:01.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.unordered_write: 0 2024-02-18T01:11:01.757 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T01:11:01.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T01:11:01.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T01:11:01.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T01:11:01.758 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.row_cache: None 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.wal_filter: None 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T01:11:01.759 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T01:11:01.760 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.max_open_files: -1 2024-02-18T01:11:01.761 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T01:11:01.762 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T01:11:01.762 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T01:11:01.762 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T01:11:01.762 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Compression algorithms supported: 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kZSTD supported: 0 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T01:11:01.763 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.624+0000 7f68b8543880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.merge_operator: 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_filter: None 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562853c91d10) 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: cache_index_and_filter_blocks: 1 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T01:11:01.764 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: pin_top_level_index_and_filter: 1 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: index_type: 0 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: data_block_index_type: 0 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: index_shortening: 1 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: hash_index_allow_collision: 1 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: checksum: 1 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: no_block_cache: 0 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_cache: 0x562853cc8f10 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_cache_name: BinnedLRUCache 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_cache_options: 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: capacity : 536870912 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: num_shard_bits : 4 2024-02-18T01:11:01.765 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: strict_capacity_limit : 0 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: high_pri_pool_ratio: 0.000 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_cache_compressed: (nil) 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: persistent_cache: (nil) 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_size: 4096 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_size_deviation: 10 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_restart_interval: 16 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: index_block_restart_interval: 1 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: metadata_block_size: 4096 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: partition_filters: 0 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: use_delta_encoding: 1 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: whole_key_filtering: 1 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: verify_compression: 0 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: read_amp_bytes_per_bit: 0 2024-02-18T01:11:01.766 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: format_version: 2 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: enable_index_compression: 1 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: block_align: 0 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression: NoCompression 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.num_levels: 7 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T01:11:01.767 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T01:11:01.768 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T01:11:01.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T01:11:01.770 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T01:11:01.771 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.ttl: 2592000 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218661631838, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218661632423, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1402, "table_properties": {"data_size": 488, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708218661, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.628+0000 7f68b8543880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218661632986, "job": 1, "event": "recovery_finished"} 2024-02-18T01:11:01.772 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f68b8543880 4 rocksdb: DB pointer 0x562853d91800 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f689f335700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f689f335700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** DB Stats ** 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:11:01.773 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** Compaction Stats [default] ** 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** Compaction Stats [default] ** 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:01.774 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** Compaction Stats [default] ** 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:11:01.775 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** Compaction Stats [default] ** 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:11:01.776 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f68b8543880 0 mon.b does not exist in monmap, will attempt to join an existing cluster 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f68b8543880 0 using public_addrv v1:172.21.15.155:6789/0 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f68b8543880 0 starting mon.b rank -1 at public addrs v1:172.21.15.155:6789/0 at bind addrs v1:172.21.15.155:6789/0 mon_data /var/lib/ceph/mon/ceph-b fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.632+0000 7f68b8543880 1 mon.b@-1(???) e0 preinit fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 0 mon.b@-1(synchronizing).mds e1 new map 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 0 mon.b@-1(synchronizing).mds e1 print_map 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: e1 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: legacy client fscid: -1 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: 2024-02-18T01:11:01.777 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: No filesystems configured 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 0 mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 0 mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T01:11:01.778 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: audit 2024-02-18T01:11:00.592082+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:01.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: audit 2024-02-18T01:11:00.594073+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:01.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: audit 2024-02-18T01:11:00.595753+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:01.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: cephadm 2024-02-18T01:11:00.597246+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi105 2024-02-18T01:11:01.779 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:01 smithi155 bash[20805]: debug 2024-02-18T01:11:01.656+0000 7f68a1b3a700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T01:11:01.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:01 smithi105 bash[19734]: audit 2024-02-18T01:11:00.592082+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:01.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:01 smithi105 bash[19734]: audit 2024-02-18T01:11:00.594073+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:01.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:01 smithi105 bash[19734]: audit 2024-02-18T01:11:00.595753+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:01.964 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:01 smithi105 bash[19734]: cephadm 2024-02-18T01:11:00.597246+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi105 2024-02-18T01:11:04.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:04 smithi105 systemd[1]: Started Ceph mon.c for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:11:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.344+0000 7f9fedbe0880 0 set uid:gid to 167:167 (ceph:ceph) 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.344+0000 7f9fedbe0880 0 ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable), process ceph-mon, pid 8 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.344+0000 7f9fedbe0880 0 pidfile_write: ignore empty --pid-file 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 0 load: jerasure load: lrc load: isa 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: RocksDB version: 6.8.1 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Compile date Feb 17 2024 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: DB SUMMARY 2024-02-18T01:11:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: CURRENT file: CURRENT 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: IDENTITY file: IDENTITY 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000003.log size: 476 ; 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.error_if_exists: 0 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.create_if_missing: 0 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.paranoid_checks: 1 2024-02-18T01:11:05.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.env: 0x556e592ff080 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.fs: Posix File System 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.info_log: 0x556e59eed980 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_file_opening_threads: 16 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.statistics: (nil) 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.use_fsync: 0 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_log_file_size: 0 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.log_file_time_to_roll: 0 2024-02-18T01:11:05.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.keep_log_file_num: 1000 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.recycle_log_file_num: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.allow_fallocate: 1 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.allow_mmap_reads: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.allow_mmap_writes: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.use_direct_reads: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.create_missing_column_families: 0 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.db_log_dir: 2024-02-18T01:11:05.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.table_cache_numshardbits: 6 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_subcompactions: 1 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_background_flushes: -1 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-02-18T01:11:05.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.advise_random_on_open: 1 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.db_write_buffer_size: 0 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.write_buffer_manager: 0x556e5abec3c0 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.use_adaptive_mutex: 0 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.rate_limiter: (nil) 2024-02-18T01:11:05.496 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-02-18T01:11:05.497 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.wal_recovery_mode: 2 2024-02-18T01:11:05.497 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.enable_thread_tracking: 0 2024-02-18T01:11:05.499 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.enable_pipelined_write: 0 2024-02-18T01:11:05.499 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.unordered_write: 0 2024-02-18T01:11:05.499 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-02-18T01:11:05.499 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-02-18T01:11:05.499 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.row_cache: None 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.wal_filter: None 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.allow_ingest_behind: 0 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.preserve_deletes: 0 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.two_write_queues: 0 2024-02-18T01:11:05.500 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.manual_wal_flush: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.atomic_flush: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.persist_stats_to_disk: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.log_readahead_size: 0 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.sst_file_checksum_func: Unknown 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_background_jobs: 2 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_background_compactions: -1 2024-02-18T01:11:05.501 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.delayed_write_rate : 16777216 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_total_wal_size: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.stats_dump_period_sec: 600 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.stats_persist_period_sec: 600 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_open_files: -1 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bytes_per_sync: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_readahead_size: 0 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Compression algorithms supported: 2024-02-18T01:11:05.502 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kZSTD supported: 0 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kXpressCompression supported: 0 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kLZ4HCCompression supported: 1 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kLZ4Compression supported: 1 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kBZip2Compression supported: 0 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kZlibCompression supported: 1 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: kSnappyCompression supported: 1 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.merge_operator: 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_filter: None 2024-02-18T01:11:05.503 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_filter_factory: None 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.table_factory: BlockBasedTable 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x556e59e9bd10) 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: cache_index_and_filter_blocks: 1 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: cache_index_and_filter_blocks_with_high_priority: 0 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: pin_top_level_index_and_filter: 1 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: index_type: 0 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: data_block_index_type: 0 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: index_shortening: 1 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: data_block_hash_table_util_ratio: 0.750000 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: hash_index_allow_collision: 1 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: checksum: 1 2024-02-18T01:11:05.504 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: no_block_cache: 0 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_cache: 0x556e59ed2f10 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_cache_name: BinnedLRUCache 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_cache_options: 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: capacity : 536870912 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: num_shard_bits : 4 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: strict_capacity_limit : 0 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: high_pri_pool_ratio: 0.000 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_cache_compressed: (nil) 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: persistent_cache: (nil) 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_size: 4096 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_size_deviation: 10 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_restart_interval: 16 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: index_block_restart_interval: 1 2024-02-18T01:11:05.505 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: metadata_block_size: 4096 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: partition_filters: 0 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: use_delta_encoding: 1 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: filter_policy: rocksdb.BuiltinBloomFilter 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: whole_key_filtering: 1 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: verify_compression: 0 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: read_amp_bytes_per_bit: 0 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: format_version: 2 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: enable_index_compression: 1 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: block_align: 0 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.write_buffer_size: 33554432 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_write_buffer_number: 2 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression: NoCompression 2024-02-18T01:11:05.506 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression: Disabled 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.prefix_extractor: nullptr 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.num_levels: 7 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.window_bits: -14 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.level: 32767 2024-02-18T01:11:05.507 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.strategy: 0 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compression_opts.enabled: false 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.target_file_size_base: 67108864 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.target_file_size_multiplier: 1 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-02-18T01:11:05.508 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.arena_block_size: 4194304 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.disable_auto_compactions: 0 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-02-18T01:11:05.509 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.table_properties_collectors: 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.inplace_update_support: 0 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.memtable_huge_page_size: 0 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.bloom_locality: 0 2024-02-18T01:11:05.510 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.max_successive_merges: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.paranoid_file_checks: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.force_consistency_checks: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.report_bg_io_stats: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.ttl: 2592000 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-02-18T01:11:05.511 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218665352806, "job": 1, "event": "recovery_started", "log_files": [3]} 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.348+0000 7f9fedbe0880 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218665353678, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1402, "table_properties": {"data_size": 488, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 366, "raw_average_value_size": 73, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1708218665, "oldest_key_time": 3, "file_creation_time": 0}} 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1708218665354598, "job": 1, "event": "recovery_finished"} 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 4 rocksdb: DB pointer 0x556e59f9b800 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fd49d2700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fd49d2700 4 rocksdb: [db_impl/db_impl.cc:851] 2024-02-18T01:11:05.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** DB Stats ** 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** Compaction Stats [default] ** 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.513 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** Compaction Stats [default] ** 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Interval compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** Compaction Stats [default] ** 2024-02-18T01:11:05.514 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: L0 1/0 1.37 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Sum 1/0 1.37 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** Compaction Stats [default] ** 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.7 0.00 0.00 1 0.001 0 0 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Uptime(secs): 0.0 total, 0.0 interval 2024-02-18T01:11:05.515 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Flush(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(GB): cumulative 0.000, interval 0.000 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(Total Files): cumulative 0, interval 0 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(L0 Files): cumulative 0, interval 0 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: AddFile(Keys): cumulative 0, interval 0 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Cumulative compaction: 0.00 GB write, 0.26 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: ** File Read Latency Histogram By Level [default] ** 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 0 mon.c does not exist in monmap, will attempt to join an existing cluster 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.352+0000 7f9fedbe0880 0 using public_addrv v1:172.21.15.105:6790/0 2024-02-18T01:11:05.516 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.356+0000 7f9fedbe0880 0 starting mon.c rank -1 at public addrs v1:172.21.15.105:6790/0 at bind addrs v1:172.21.15.105:6790/0 mon_data /var/lib/ceph/mon/ceph-c fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.356+0000 7f9fedbe0880 1 mon.c@-1(???) e0 preinit fsid 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 0 mon.c@-1(synchronizing).mds e1 new map 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 0 mon.c@-1(synchronizing).mds e1 print_map 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: e1 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: enable_multiple, ever_enabled_multiple: 1,1 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: legacy client fscid: -1 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: No filesystems configured 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-02-18T01:11:05.517 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.380+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.384+0000 7f9fd71d7700 0 mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.384+0000 7f9fd71d7700 0 mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: audit 2024-02-18T01:11:00.592082+0000 mon.a (mon.0) 128 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: audit 2024-02-18T01:11:00.594073+0000 mon.a (mon.0) 129 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: audit 2024-02-18T01:11:00.595753+0000 mon.a (mon.0) 130 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: cephadm 2024-02-18T01:11:00.597246+0000 mgr.y (mgr.14146) 39 : cephadm [INF] Deploying daemon mon.c on smithi105 2024-02-18T01:11:05.518 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:05 smithi105 bash[27669]: debug 2024-02-18T01:11:05.384+0000 7f9fd71d7700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-02-18T01:11:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- 172.21.15.155:0/2245940810 >> v1:172.21.15.105:6789/0 conn(0x7fcd180fd520 legacy=0x7fcd180fd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- 172.21.15.155:0/2245940810 shutdown_connections 2024-02-18T01:11:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- 172.21.15.155:0/2245940810 >> 172.21.15.155:0/2245940810 conn(0x7fcd1806c300 msgr2=0x7fcd1806c710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- 172.21.15.155:0/2245940810 shutdown_connections 2024-02-18T01:11:06.723 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- 172.21.15.155:0/2245940810 wait complete. 2024-02-18T01:11:06.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 Processor -- start 2024-02-18T01:11:06.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.720+0000 7fcd1cf55700 1 -- start start 2024-02-18T01:11:06.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1cf55700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcd18112ff0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1cf55700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcd18114190 con 0x7fcd180fd520 2024-02-18T01:11:06.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1659c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fcd180fd520 0x7fcd1810cab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:58762/0 (socket says 172.21.15.155:58762) 2024-02-18T01:11:06.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1659c700 1 -- 172.21.15.155:0/3849470523 learned_addr learned my addr 172.21.15.155:0/3849470523 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:11:06.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725822928 0 0) 0x7fcd18112ff0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd000053b0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 668660121 0 0) 0x7fcd000053b0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcd18112ff0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.728 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 3280120948 0 0) 0x7fcd08001800 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 382221114 0 0) 0x7fcd18112ff0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 >> v1:172.21.15.155:6789/0 conn(0x7fcd180fd520 legacy=0x7fcd1810cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd18115330 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd181131e0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd181136d0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fcd080023a0 con 0x7fcd1810d1c0 2024-02-18T01:11:06.729 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.724+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 239+0+0 (unknown 3280120948 0 0) 0x7fcd08003230 con 0x7fcd1810d1c0 2024-02-18T01:11:06.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.728+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fcd0800cb70 con 0x7fcd1810d1c0 2024-02-18T01:11:06.730 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.728+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7fcd0803c120 con 0x7fcd1810d1c0 2024-02-18T01:11:06.731 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.728+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccf8004f50 con 0x7fcd1810d1c0 2024-02-18T01:11:06.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:06.736+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fcd0800e3b0 con 0x7fcd1810d1c0 2024-02-18T01:11:07.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fccf8005200 con 0x7fcd1810d1c0 2024-02-18T01:11:07.002 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd077fe700 1 -- 172.21.15.155:0/3849470523 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+893 (unknown 3557084514 0 2489562019) 0x7fcd08010490 con 0x7fcd1810d1c0 2024-02-18T01:11:07.003 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-18T01:11:07.003 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":2,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","modified":"2024-02-18T01:11:01.662909Z","created":"2024-02-18T01:08:53.400007Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 >> v1:172.21.15.105:6800/3087430119 conn(0x7fcd00033a70 legacy=0x7fcd00035f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 >> v1:172.21.15.105:6789/0 conn(0x7fcd1810d1c0 legacy=0x7fcd181118d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 shutdown_connections 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 >> 172.21.15.155:0/3849470523 conn(0x7fcd1806c300 msgr2=0x7fcd180f5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 shutdown_connections 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:07.000+0000 7fcd1cf55700 1 -- 172.21.15.155:0/3849470523 wait complete. 2024-02-18T01:11:07.008 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 2 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:01.458563+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:01.666009+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:01.666418+0000 mon.a (mon.0) 133 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:01.686598+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:02.662936+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:03.458997+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:03.663192+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:03.665568+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:07.062 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:04.663401+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:05.393260+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:05.459411+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:05.663412+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:06.393072+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:06.663547+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.707004+0000 mon.a (mon.0) 142 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.714322+0000 mon.a (mon.0) 143 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.715461+0000 mon.a (mon.0) 144 : cluster [DBG] fsmap 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.715525+0000 mon.a (mon.0) 145 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.715861+0000 mon.a (mon.0) 146 : cluster [DBG] mgrmap e13: y(active, since 71s) 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: cluster 2024-02-18T01:11:06.716149+0000 mon.a (mon.0) 147 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:06.724558+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:07.063 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:06 smithi155 bash[20805]: audit 2024-02-18T01:11:06.727274+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:01.458563+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:01.666009+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:01.666418+0000 mon.a (mon.0) 133 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:01.686598+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:02.662936+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:03.458997+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:03.663192+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:03.665568+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:04.663401+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:05.393260+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:05.459411+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:05.663412+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:06.393072+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:06.663547+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.707004+0000 mon.a (mon.0) 142 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.714322+0000 mon.a (mon.0) 143 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.715461+0000 mon.a (mon.0) 144 : cluster [DBG] fsmap 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.715525+0000 mon.a (mon.0) 145 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.715861+0000 mon.a (mon.0) 146 : cluster [DBG] mgrmap e13: y(active, since 71s) 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: cluster 2024-02-18T01:11:06.716149+0000 mon.a (mon.0) 147 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:06.724558+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:07.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:06 smithi105 bash[19734]: audit 2024-02-18T01:11:06.727274+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:07.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:11:07 smithi105 bash[20046]: debug 2024-02-18T01:11:07.660+0000 7f86a7d4e700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-02-18T01:11:08.667 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-02-18T01:11:08.667 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph mon dump -f json 2024-02-18T01:11:12.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- 172.21.15.155:0/1489629334 >> v1:172.21.15.105:6789/0 conn(0x7fde6006b0f0 legacy=0x7fde6006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:12.432 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- 172.21.15.155:0/1489629334 shutdown_connections 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- 172.21.15.155:0/1489629334 >> 172.21.15.155:0/1489629334 conn(0x7fde600f30f0 msgr2=0x7fde600f5510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- 172.21.15.155:0/1489629334 shutdown_connections 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- 172.21.15.155:0/1489629334 wait complete. 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 Processor -- start 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- start start 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde60119170 con 0x7fde6010f650 2024-02-18T01:11:12.433 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde6011a370 con 0x7fde6006b0f0 2024-02-18T01:11:12.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.428+0000 7fde64edd700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fde6011b570 con 0x7fde60115480 2024-02-18T01:11:12.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde5e59c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fde6006b0f0 0x7fde6010ef40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:52560/0 (socket says 172.21.15.155:52560) 2024-02-18T01:11:12.434 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde5e59c700 1 -- 172.21.15.155:0/193737720 learned_addr learned my addr 172.21.15.155:0/193737720 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:11:12.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420718406 0 0) 0x7fde6011b570 con 0x7fde60115480 2024-02-18T01:11:12.436 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde4c003ae0 con 0x7fde60115480 2024-02-18T01:11:12.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2051004769 0 0) 0x7fde4c003ae0 con 0x7fde60115480 2024-02-18T01:11:12.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde6011b570 con 0x7fde60115480 2024-02-18T01:11:12.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.432+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fde500017c0 con 0x7fde60115480 2024-02-18T01:11:12.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1225475057 0 0) 0x7fde60119170 con 0x7fde6010f650 2024-02-18T01:11:12.437 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde4c003ae0 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4076147759 0 0) 0x7fde4c003ae0 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fde60119170 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fde480017e0 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 197357543 0 0) 0x7fde60119170 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 >> v1:172.21.15.105:6790/0 conn(0x7fde60115480 legacy=0x7fde60117870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 >> v1:172.21.15.155:6789/0 conn(0x7fde6006b0f0 legacy=0x7fde6010ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde6011c770 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde6011b760 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fde6011bbd0 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fde48002420 con 0x7fde6010f650 2024-02-18T01:11:12.439 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fde480032d0 con 0x7fde6010f650 2024-02-18T01:11:12.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fde4800cb50 con 0x7fde6010f650 2024-02-18T01:11:12.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7fde4803d030 con 0x7fde6010f650 2024-02-18T01:11:12.440 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.436+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40004f20 con 0x7fde6010f650 2024-02-18T01:11:12.451 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.448+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fde4800e400 con 0x7fde6010f650 2024-02-18T01:11:12.709 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.704+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fde400051d0 con 0x7fde6010f650 2024-02-18T01:11:12.710 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde577fe700 1 -- 172.21.15.155:0/193737720 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1111 (unknown 409899127 0 2523122329) 0x7fde480104e0 con 0x7fde6010f650 2024-02-18T01:11:12.710 INFO:teuthology.orchestra.run.smithi155.stdout: 2024-02-18T01:11:12.710 INFO:teuthology.orchestra.run.smithi155.stdout:{"epoch":3,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","modified":"2024-02-18T01:11:07.394911Z","created":"2024-02-18T01:08:53.400007Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6789","nonce":0}]},"addr":"172.21.15.105:6789/0","public_addr":"172.21.15.105:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6789","nonce":0}]},"addr":"172.21.15.155:6789/0","public_addr":"172.21.15.155:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6790","nonce":0}]},"addr":"172.21.15.105:6790/0","public_addr":"172.21.15.105:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 >> v1:172.21.15.105:6800/3087430119 conn(0x7fde4c033a70 legacy=0x7fde4c035f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 >> v1:172.21.15.105:6789/0 conn(0x7fde6010f650 legacy=0x7fde60113d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 shutdown_connections 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 >> 172.21.15.155:0/193737720 conn(0x7fde600f30f0 msgr2=0x7fde600fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 shutdown_connections 2024-02-18T01:11:12.712 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:12.708+0000 7fde64edd700 1 -- 172.21.15.155:0/193737720 wait complete. 2024-02-18T01:11:12.713 INFO:teuthology.orchestra.run.smithi155.stderr:dumped monmap epoch 3 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:01.458563+0000 mgr.y (mgr.14146) 40 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:01.666009+0000 mon.a (mon.0) 132 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:01.666418+0000 mon.a (mon.0) 133 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:01.686598+0000 mon.a (mon.0) 134 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:02.662936+0000 mon.a (mon.0) 135 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:03.458997+0000 mgr.y (mgr.14146) 41 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:03.663192+0000 mon.a (mon.0) 136 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:03.665568+0000 mon.b (mon.1) 1 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:04.663401+0000 mon.a (mon.0) 137 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:05.393260+0000 mon.a (mon.0) 138 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:05.459411+0000 mgr.y (mgr.14146) 42 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:05.663412+0000 mon.a (mon.0) 139 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:06.393072+0000 mon.a (mon.0) 140 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:06.663547+0000 mon.a (mon.0) 141 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.707004+0000 mon.a (mon.0) 142 : cluster [INF] mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.714322+0000 mon.a (mon.0) 143 : cluster [DBG] monmap e2: 2 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0} removed_ranks: {} 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.715461+0000 mon.a (mon.0) 144 : cluster [DBG] fsmap 2024-02-18T01:11:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.715525+0000 mon.a (mon.0) 145 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.715861+0000 mon.a (mon.0) 146 : cluster [DBG] mgrmap e13: y(active, since 71s) 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:06.716149+0000 mon.a (mon.0) 147 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:06.724558+0000 mon.a (mon.0) 148 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:06.727274+0000 mon.a (mon.0) 149 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:07.401094+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:07.401571+0000 mon.a (mon.0) 153 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:07.403518+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:07.403769+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:07.407595+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:07.459822+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:08.393545+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:09.393532+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:09.396527+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:09.460312+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:10.393749+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:11.393798+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:11.460751+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:12.393992+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.407063+0000 mon.a (mon.0) 161 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.421245+0000 mon.a (mon.0) 162 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.105:6790/0} removed_ranks: {} 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.422911+0000 mon.a (mon.0) 163 : cluster [DBG] fsmap 2024-02-18T01:11:12.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.423011+0000 mon.a (mon.0) 164 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.423428+0000 mon.a (mon.0) 165 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: cluster 2024-02-18T01:11:12.423817+0000 mon.a (mon.0) 166 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:12.430294+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:12.436191+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[27669]: audit 2024-02-18T01:11:12.441971+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:07.401094+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:07.401571+0000 mon.a (mon.0) 153 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:07.403518+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:07.403769+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:07.407595+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:07.459822+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:08.393545+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:09.393532+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:09.396527+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:09.460312+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:10.393749+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:11.393798+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:11.460751+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:12.393992+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.407063+0000 mon.a (mon.0) 161 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.421245+0000 mon.a (mon.0) 162 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.105:6790/0} removed_ranks: {} 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.422911+0000 mon.a (mon.0) 163 : cluster [DBG] fsmap 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.423011+0000 mon.a (mon.0) 164 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.423428+0000 mon.a (mon.0) 165 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: cluster 2024-02-18T01:11:12.423817+0000 mon.a (mon.0) 166 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:12.430294+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:12.436191+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:12 smithi105 bash[19734]: audit 2024-02-18T01:11:12.441971+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:07.401094+0000 mon.a (mon.0) 152 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:07.401571+0000 mon.a (mon.0) 153 : cluster [INF] mon.a calling monitor election 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:07.403518+0000 mon.a (mon.0) 154 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:07.403769+0000 mon.a (mon.0) 155 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:07.407595+0000 mon.b (mon.1) 2 : cluster [INF] mon.b calling monitor election 2024-02-18T01:11:12.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:07.459822+0000 mgr.y (mgr.14146) 43 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:08.393545+0000 mon.a (mon.0) 156 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:09.393532+0000 mon.a (mon.0) 157 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:09.396527+0000 mon.c (mon.2) 1 : cluster [INF] mon.c calling monitor election 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:09.460312+0000 mgr.y (mgr.14146) 44 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:10.393749+0000 mon.a (mon.0) 158 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:11.393798+0000 mon.a (mon.0) 159 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:11.460751+0000 mgr.y (mgr.14146) 45 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:12.393992+0000 mon.a (mon.0) 160 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.407063+0000 mon.a (mon.0) 161 : cluster [INF] mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.421245+0000 mon.a (mon.0) 162 : cluster [DBG] monmap e3: 3 mons at {a=v1:172.21.15.105:6789/0,b=v1:172.21.15.155:6789/0,c=v1:172.21.15.105:6790/0} removed_ranks: {} 2024-02-18T01:11:12.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.422911+0000 mon.a (mon.0) 163 : cluster [DBG] fsmap 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.423011+0000 mon.a (mon.0) 164 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.423428+0000 mon.a (mon.0) 165 : cluster [DBG] mgrmap e13: y(active, since 76s) 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: cluster 2024-02-18T01:11:12.423817+0000 mon.a (mon.0) 166 : cluster [INF] overall HEALTH_OK 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:12.430294+0000 mon.a (mon.0) 167 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:12.436191+0000 mon.a (mon.0) 168 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:12.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:12 smithi155 bash[20805]: audit 2024-02-18T01:11:12.441971+0000 mon.a (mon.0) 169 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.397 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-02-18T01:11:13.397 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph config generate-minimal-conf 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.709252+0000 mon.a (mon.0) 170 : audit [DBG] from='client.? v1:172.21.15.155:0/193737720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.731268+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.732765+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: cephadm 2024-02-18T01:11:12.734610+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: cephadm 2024-02-18T01:11:12.734972+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.760732+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.765348+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.770307+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: cephadm 2024-02-18T01:11:12.772347+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.772770+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.773836+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:12.774834+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: cephadm 2024-02-18T01:11:12.775751+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi105 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[27669]: audit 2024-02-18T01:11:13.394226+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.709252+0000 mon.a (mon.0) 170 : audit [DBG] from='client.? v1:172.21.15.155:0/193737720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.731268+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.732765+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: cephadm 2024-02-18T01:11:12.734610+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: cephadm 2024-02-18T01:11:12.734972+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.760732+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.765348+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.770307+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: cephadm 2024-02-18T01:11:12.772347+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T01:11:13.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.772770+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:13.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.773836+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:13.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:12.774834+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: cephadm 2024-02-18T01:11:12.775751+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi105 2024-02-18T01:11:13.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:13 smithi105 bash[19734]: audit 2024-02-18T01:11:13.394226+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.709252+0000 mon.a (mon.0) 170 : audit [DBG] from='client.? v1:172.21.15.155:0/193737720' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-02-18T01:11:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.731268+0000 mon.a (mon.0) 171 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.732765+0000 mon.a (mon.0) 172 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: cephadm 2024-02-18T01:11:12.734610+0000 mgr.y (mgr.14146) 46 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:11:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: cephadm 2024-02-18T01:11:12.734972+0000 mgr.y (mgr.14146) 47 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.760732+0000 mon.a (mon.0) 173 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.765348+0000 mon.a (mon.0) 174 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.770307+0000 mon.a (mon.0) 175 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: cephadm 2024-02-18T01:11:12.772347+0000 mgr.y (mgr.14146) 48 : cephadm [INF] Reconfiguring mon.a (unknown last config time)... 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.772770+0000 mon.a (mon.0) 176 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.773836+0000 mon.a (mon.0) 177 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:12.774834+0000 mon.a (mon.0) 178 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: cephadm 2024-02-18T01:11:12.775751+0000 mgr.y (mgr.14146) 49 : cephadm [INF] Reconfiguring daemon mon.a on smithi105 2024-02-18T01:11:13.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:13 smithi155 bash[20805]: audit 2024-02-18T01:11:13.394226+0000 mon.a (mon.0) 179 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:11:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:14 smithi105 bash[19734]: cluster 2024-02-18T01:11:13.461256+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:14 smithi105 bash[27669]: cluster 2024-02-18T01:11:13.461256+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:14 smithi155 bash[20805]: cluster 2024-02-18T01:11:13.461256+0000 mgr.y (mgr.14146) 50 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: cluster 2024-02-18T01:11:15.461682+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: audit 2024-02-18T01:11:15.600252+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: cephadm 2024-02-18T01:11:15.601145+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: audit 2024-02-18T01:11:15.601516+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: audit 2024-02-18T01:11:15.602433+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: audit 2024-02-18T01:11:15.603598+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:16.859 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[27669]: cephadm 2024-02-18T01:11:15.604661+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring daemon mon.c on smithi105 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: cluster 2024-02-18T01:11:15.461682+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: audit 2024-02-18T01:11:15.600252+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: cephadm 2024-02-18T01:11:15.601145+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: audit 2024-02-18T01:11:15.601516+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: audit 2024-02-18T01:11:15.602433+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: audit 2024-02-18T01:11:15.603598+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:16.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:16 smithi105 bash[19734]: cephadm 2024-02-18T01:11:15.604661+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring daemon mon.c on smithi105 2024-02-18T01:11:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: cluster 2024-02-18T01:11:15.461682+0000 mgr.y (mgr.14146) 51 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: audit 2024-02-18T01:11:15.600252+0000 mon.a (mon.0) 180 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: cephadm 2024-02-18T01:11:15.601145+0000 mgr.y (mgr.14146) 52 : cephadm [INF] Reconfiguring mon.c (monmap changed)... 2024-02-18T01:11:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: audit 2024-02-18T01:11:15.601516+0000 mon.a (mon.0) 181 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: audit 2024-02-18T01:11:15.602433+0000 mon.a (mon.0) 182 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: audit 2024-02-18T01:11:15.603598+0000 mon.a (mon.0) 183 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:16 smithi155 bash[20805]: cephadm 2024-02-18T01:11:15.604661+0000 mgr.y (mgr.14146) 53 : cephadm [INF] Reconfiguring daemon mon.c on smithi105 2024-02-18T01:11:17.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/963470439 >> v1:172.21.15.105:6790/0 conn(0x7fb9ec0ff3d0 legacy=0x7fb9ec1017c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/963470439 shutdown_connections 2024-02-18T01:11:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/963470439 >> 172.21.15.105:0/963470439 conn(0x7fb9ec0f21b0 msgr2=0x7fb9ec0f45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/963470439 shutdown_connections 2024-02-18T01:11:17.112 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/963470439 wait complete. 2024-02-18T01:11:17.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 Processor -- start 2024-02-18T01:11:17.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- start start 2024-02-18T01:11:17.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb9ec119470 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb9ec11a670 con 0x7fb9ec102f10 2024-02-18T01:11:17.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9f0f5b700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb9ec11b870 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9eaffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb9ec0ff3d0 0x7fb9ec10ac30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46554/0 (socket says 172.21.15.105:46554) 2024-02-18T01:11:17.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.108+0000 7fb9eaffd700 1 -- 172.21.15.105:0/4096347827 learned_addr learned my addr 172.21.15.105:0/4096347827 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076623498 0 0) 0x7fb9ec119470 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d40033f0 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887445947 0 0) 0x7fb9ec11b870 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec119470 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648921544 0 0) 0x7fb9ec11a670 con 0x7fb9ec102f10 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9ec11b870 con 0x7fb9ec102f10 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3358932910 0 0) 0x7fb9d40033f0 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb9ec11a670 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb9e0002d20 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1101203806 0 0) 0x7fb9ec119470 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb9d40033f0 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb9dc0018c0 con 0x7fb9ec0fc5d0 2024-02-18T01:11:17.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 272606027 0 0) 0x7fb9ec11a670 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 >> v1:172.21.15.105:6790/0 conn(0x7fb9ec0fc5d0 legacy=0x7fb9ec10a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 >> v1:172.21.15.155:6789/0 conn(0x7fb9ec102f10 legacy=0x7fb9ec117b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9ec11ca70 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9ec119660 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb9e0001b30 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb9e0003920 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9ec119b50 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fb9e0003b60 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.112+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7fb9e0002520 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.116+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9ec05f0b0 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.125 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.120+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb9e0011050 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.309 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb9ec046ce0 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.310 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9e8ff9700 1 -- 172.21.15.105:0/4096347827 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+193 (unknown 2189217731 0 2770792260) 0x7fb9e0014830 con 0x7fb9ec0ff3d0 2024-02-18T01:11:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:# minimal ceph.conf for 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:17.310 INFO:teuthology.orchestra.run.smithi105.stdout:[global] 2024-02-18T01:11:17.310 INFO:teuthology.orchestra.run.smithi105.stdout: fsid = 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:11:17.310 INFO:teuthology.orchestra.run.smithi105.stdout: mon_host = 172.21.15.105:6789/0 172.21.15.155:6789/0 v1:172.21.15.105:6790/0 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb9d4033700 legacy=0x7fb9d4035bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 >> v1:172.21.15.105:6789/0 conn(0x7fb9ec0ff3d0 legacy=0x7fb9ec10ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 shutdown_connections 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 >> 172.21.15.105:0/4096347827 conn(0x7fb9ec0f21b0 msgr2=0x7fb9ec060fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 shutdown_connections 2024-02-18T01:11:17.312 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:17.308+0000 7fb9f0f5b700 1 -- 172.21.15.105:0/4096347827 wait complete. 2024-02-18T01:11:17.760 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:17 smithi105 bash[19734]: audit 2024-02-18T01:11:17.309749+0000 mon.a (mon.0) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/4096347827' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:17.761 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:17 smithi105 bash[27669]: audit 2024-02-18T01:11:17.309749+0000 mon.a (mon.0) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/4096347827' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:17.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:17 smithi155 bash[20805]: audit 2024-02-18T01:11:17.309749+0000 mon.a (mon.0) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/4096347827' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:18.169 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-02-18T01:11:18.170 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:11:18.170 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T01:11:18.184 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:11:18.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:11:18.243 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:11:18.243 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.conf 2024-02-18T01:11:18.259 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:11:18.259 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:11:18.311 INFO:tasks.cephadm:Adding mgr.y on smithi105 2024-02-18T01:11:18.311 INFO:tasks.cephadm:Adding mgr.x on smithi155 2024-02-18T01:11:18.311 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply mgr '2;smithi105=y;smithi155=x' 2024-02-18T01:11:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: cluster 2024-02-18T01:11:17.462077+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: audit 2024-02-18T01:11:18.054974+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: cephadm 2024-02-18T01:11:18.056032+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: audit 2024-02-18T01:11:18.056573+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: audit 2024-02-18T01:11:18.057937+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: audit 2024-02-18T01:11:18.059256+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[27669]: cephadm 2024-02-18T01:11:18.060326+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring daemon mon.b on smithi155 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: cluster 2024-02-18T01:11:17.462077+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: audit 2024-02-18T01:11:18.054974+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: cephadm 2024-02-18T01:11:18.056032+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: audit 2024-02-18T01:11:18.056573+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: audit 2024-02-18T01:11:18.057937+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: audit 2024-02-18T01:11:18.059256+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:19.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:19 smithi105 bash[19734]: cephadm 2024-02-18T01:11:18.060326+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring daemon mon.b on smithi155 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: cluster 2024-02-18T01:11:17.462077+0000 mgr.y (mgr.14146) 54 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: audit 2024-02-18T01:11:18.054974+0000 mon.a (mon.0) 185 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: cephadm 2024-02-18T01:11:18.056032+0000 mgr.y (mgr.14146) 55 : cephadm [INF] Reconfiguring mon.b (monmap changed)... 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: audit 2024-02-18T01:11:18.056573+0000 mon.a (mon.0) 186 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: audit 2024-02-18T01:11:18.057937+0000 mon.a (mon.0) 187 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: audit 2024-02-18T01:11:18.059256+0000 mon.a (mon.0) 188 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:19 smithi155 bash[20805]: cephadm 2024-02-18T01:11:18.060326+0000 mgr.y (mgr.14146) 56 : cephadm [INF] Reconfiguring daemon mon.b on smithi155 2024-02-18T01:11:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: cluster 2024-02-18T01:11:19.462535+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: audit 2024-02-18T01:11:20.917093+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: audit 2024-02-18T01:11:20.919194+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: audit 2024-02-18T01:11:20.922265+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: audit 2024-02-18T01:11:20.923864+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[27669]: audit 2024-02-18T01:11:20.932946+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: cluster 2024-02-18T01:11:19.462535+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: audit 2024-02-18T01:11:20.917093+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: audit 2024-02-18T01:11:20.919194+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: audit 2024-02-18T01:11:20.922265+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: audit 2024-02-18T01:11:20.923864+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:21 smithi105 bash[19734]: audit 2024-02-18T01:11:20.932946+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: cluster 2024-02-18T01:11:19.462535+0000 mgr.y (mgr.14146) 57 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: audit 2024-02-18T01:11:20.917093+0000 mon.a (mon.0) 189 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: audit 2024-02-18T01:11:20.919194+0000 mon.a (mon.0) 190 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: audit 2024-02-18T01:11:20.922265+0000 mon.a (mon.0) 191 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: audit 2024-02-18T01:11:20.923864+0000 mon.a (mon.0) 192 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:21.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:21 smithi155 bash[20805]: audit 2024-02-18T01:11:20.932946+0000 mon.a (mon.0) 193 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:21.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- 172.21.15.155:0/3841787456 >> v1:172.21.15.155:6789/0 conn(0x7fa7740fabf0 legacy=0x7fa7740fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:21.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- 172.21.15.155:0/3841787456 shutdown_connections 2024-02-18T01:11:21.919 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- 172.21.15.155:0/3841787456 >> 172.21.15.155:0/3841787456 conn(0x7fa7740f1e80 msgr2=0x7fa7740f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:21.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- 172.21.15.155:0/3841787456 shutdown_connections 2024-02-18T01:11:21.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- 172.21.15.155:0/3841787456 wait complete. 2024-02-18T01:11:21.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 Processor -- start 2024-02-18T01:11:21.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- start start 2024-02-18T01:11:21.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa774114c10 con 0x7fa7740f6720 2024-02-18T01:11:21.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa774115e10 con 0x7fa7740fabf0 2024-02-18T01:11:21.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa779d65700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa774117010 con 0x7fa7740fe820 2024-02-18T01:11:21.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa772ffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa7740fabf0 0x7fa77410f9a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:52586/0 (socket says 172.21.15.155:52586) 2024-02-18T01:11:21.921 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.915+0000 7fa772ffd700 1 -- 172.21.15.155:0/347113267 learned_addr learned my addr 172.21.15.155:0/347113267 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736089984 0 0) 0x7fa774115e10 con 0x7fa7740fabf0 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa75c0033f0 con 0x7fa7740fabf0 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 254608978 0 0) 0x7fa774117010 con 0x7fa7740fe820 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa774115e10 con 0x7fa7740fe820 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3313040119 0 0) 0x7fa774114c10 con 0x7fa7740f6720 2024-02-18T01:11:21.922 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa774117010 con 0x7fa7740f6720 2024-02-18T01:11:21.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4150548468 0 0) 0x7fa75c0033f0 con 0x7fa7740fabf0 2024-02-18T01:11:21.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa774114c10 con 0x7fa7740fabf0 2024-02-18T01:11:21.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa768002ec0 con 0x7fa7740fabf0 2024-02-18T01:11:21.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 771738142 0 0) 0x7fa774115e10 con 0x7fa7740fe820 2024-02-18T01:11:21.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa75c0033f0 con 0x7fa7740fe820 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa76c001b70 con 0x7fa7740fe820 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3769303971 0 0) 0x7fa774117010 con 0x7fa7740f6720 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa774115e10 con 0x7fa7740f6720 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2192361243 0 0) 0x7fa774114c10 con 0x7fa7740fabf0 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 >> v1:172.21.15.105:6790/0 conn(0x7fa7740fe820 legacy=0x7fa7741132d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 >> v1:172.21.15.105:6789/0 conn(0x7fa7740f6720 legacy=0x7fa77410c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:21.924 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa774118210 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa774117200 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7741176f0 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fa7680011c0 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa768003920 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7fa768003b20 con 0x7fa7740fabf0 2024-02-18T01:11:21.925 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.919+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7fa76803fd20 con 0x7fa7740fabf0 2024-02-18T01:11:21.926 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.923+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa760004f20 con 0x7fa7740fabf0 2024-02-18T01:11:21.932 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:21.927+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa7680111f0 con 0x7fa7740fabf0 2024-02-18T01:11:22.123 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.119+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi105=y;smithi155=x", "target": ["mon-mgr", ""]}) v1 -- 0x7fa760000b90 con 0x7fa75c033a60 2024-02-18T01:11:22.135 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.131+0000 7fa770ff9700 1 -- 172.21.15.155:0/347113267 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (unknown 0 0 325935098) 0x7fa760000b90 con 0x7fa75c033a60 2024-02-18T01:11:22.136 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled mgr update... 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 >> v1:172.21.15.105:6800/3087430119 conn(0x7fa75c033a60 legacy=0x7fa75c035f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 >> v1:172.21.15.155:6789/0 conn(0x7fa7740fabf0 legacy=0x7fa77410f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 shutdown_connections 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 >> 172.21.15.155:0/347113267 conn(0x7fa7740f1e80 msgr2=0x7fa774060a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 shutdown_connections 2024-02-18T01:11:22.138 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:11:22.135+0000 7fa779d65700 1 -- 172.21.15.155:0/347113267 wait complete. 2024-02-18T01:11:22.851 DEBUG:teuthology.orchestra.run.smithi155:mgr.x> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.x.service 2024-02-18T01:11:22.854 INFO:tasks.cephadm:Deploying OSDs... 2024-02-18T01:11:22.854 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:11:22.854 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T01:11:22.863 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T01:11:22.863 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_1 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 342 Links: 1 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-02-18 01:04:53.856637230 +0000 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-02-18 01:04:53.852637325 +0000 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-02-18 01:04:53.852637325 +0000 2024-02-18T01:11:22.912 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-02-18T01:11:22.913 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T01:11:22.967 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-02-18T01:11:22.967 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-02-18T01:11:22.967 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000417024 s, 1.2 MB/s 2024-02-18T01:11:22.969 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T01:11:23.021 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_2 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 348 Links: 1 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-02-18 01:04:54.308626479 +0000 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-02-18 01:04:54.156630094 +0000 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-02-18 01:04:54.156630094 +0000 2024-02-18T01:11:23.074 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-02-18T01:11:23.074 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T01:11:23.129 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-02-18T01:11:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-02-18T01:11:23.130 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000384484 s, 1.3 MB/s 2024-02-18T01:11:23.131 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T01:11:23.182 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_3 2024-02-18T01:11:23.222 INFO:journalctl@ceph.mgr.x.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:11:23.231 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T01:11:23.231 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.231 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 354 Links: 1 2024-02-18T01:11:23.232 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.232 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-02-18 01:04:54.596619629 +0000 2024-02-18T01:11:23.232 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-02-18 01:04:54.412624005 +0000 2024-02-18T01:11:23.232 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-02-18 01:04:54.412624005 +0000 2024-02-18T01:11:23.232 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-02-18T01:11:23.232 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T01:11:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: cluster 2024-02-18T01:11:21.462945+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.134477+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.153404+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.157316+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.159205+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.167692+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.170439+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.175069+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.176561+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[27669]: audit 2024-02-18T01:11:22.178045+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: cluster 2024-02-18T01:11:21.462945+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.134477+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.153404+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.157316+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.159205+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.167692+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.170439+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.175069+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.176561+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:23.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:23 smithi105 bash[19734]: audit 2024-02-18T01:11:22.178045+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: cluster 2024-02-18T01:11:21.462945+0000 mgr.y (mgr.14146) 58 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.134477+0000 mon.a (mon.0) 194 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.153404+0000 mon.a (mon.0) 195 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.157316+0000 mon.a (mon.0) 196 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.159205+0000 mon.a (mon.0) 197 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.167692+0000 mon.a (mon.0) 198 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.170439+0000 mon.a (mon.0) 199 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.175069+0000 mon.a (mon.0) 200 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.176561+0000 mon.a (mon.0) 201 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:23 smithi155 bash[20805]: audit 2024-02-18T01:11:22.178045+0000 mon.a (mon.0) 202 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-02-18T01:11:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-02-18T01:11:23.254 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000333346 s, 1.5 MB/s 2024-02-18T01:11:23.255 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T01:11:23.305 DEBUG:teuthology.orchestra.run.smithi105:> stat /dev/vg_nvme/lv_4 2024-02-18T01:11:23.356 INFO:teuthology.orchestra.run.smithi105.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T01:11:23.356 INFO:teuthology.orchestra.run.smithi105.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.356 INFO:teuthology.orchestra.run.smithi105.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-02-18T01:11:23.356 INFO:teuthology.orchestra.run.smithi105.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.356 INFO:teuthology.orchestra.run.smithi105.stdout:Access: 2024-02-18 01:04:54.904612303 +0000 2024-02-18T01:11:23.357 INFO:teuthology.orchestra.run.smithi105.stdout:Modify: 2024-02-18 01:04:54.728616489 +0000 2024-02-18T01:11:23.357 INFO:teuthology.orchestra.run.smithi105.stdout:Change: 2024-02-18 01:04:54.728616489 +0000 2024-02-18T01:11:23.357 INFO:teuthology.orchestra.run.smithi105.stdout: Birth: - 2024-02-18T01:11:23.357 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T01:11:23.410 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records in 2024-02-18T01:11:23.410 INFO:teuthology.orchestra.run.smithi105.stderr:1+0 records out 2024-02-18T01:11:23.411 INFO:teuthology.orchestra.run.smithi105.stderr:512 bytes copied, 0.000311444 s, 1.6 MB/s 2024-02-18T01:11:23.412 DEBUG:teuthology.orchestra.run.smithi105:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T01:11:23.462 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:11:23.462 DEBUG:teuthology.orchestra.run.smithi155:> dd if=/scratch_devs of=/dev/stdout 2024-02-18T01:11:23.470 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-02-18T01:11:23.470 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_1 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 345 Links: 1 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-18 01:04:55.059742872 +0000 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-18 01:04:54.891750843 +0000 2024-02-18T01:11:23.520 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-18 01:04:54.891750843 +0000 2024-02-18T01:11:23.521 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-02-18T01:11:23.521 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-02-18T01:11:23.570 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-18T01:11:23.570 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-18T01:11:23.570 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000324041 s, 1.6 MB/s 2024-02-18T01:11:23.571 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-02-18T01:11:23.622 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_2 2024-02-18T01:11:23.677 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-02-18T01:11:23.677 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.677 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 352 Links: 1 2024-02-18T01:11:23.677 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.678 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-18 01:04:55.339729775 +0000 2024-02-18T01:11:23.678 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-18 01:04:55.215735578 +0000 2024-02-18T01:11:23.678 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-18 01:04:55.215735578 +0000 2024-02-18T01:11:23.678 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-02-18T01:11:23.678 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-02-18T01:11:23.734 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-18T01:11:23.735 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-18T01:11:23.735 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000376771 s, 1.4 MB/s 2024-02-18T01:11:23.736 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-02-18T01:11:23.786 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_3 2024-02-18T01:11:23.834 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-02-18T01:11:23.834 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.834 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 358 Links: 1 2024-02-18T01:11:23.834 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.834 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-18 01:04:55.639715736 +0000 2024-02-18T01:11:23.835 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-18 01:04:55.487722849 +0000 2024-02-18T01:11:23.835 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-18 01:04:55.487722849 +0000 2024-02-18T01:11:23.835 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-02-18T01:11:23.835 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-02-18T01:11:23.891 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-18T01:11:23.891 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-18T01:11:23.891 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000391561 s, 1.3 MB/s 2024-02-18T01:11:23.892 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-02-18T01:11:23.941 DEBUG:teuthology.orchestra.run.smithi155:> stat /dev/vg_nvme/lv_4 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout:Device: 6h/6d Inode: 364 Links: 1 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout:Access: 2024-02-18 01:04:55.923702446 +0000 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout:Modify: 2024-02-18 01:04:55.755710307 +0000 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout:Change: 2024-02-18 01:04:55.755710307 +0000 2024-02-18T01:11:23.994 INFO:teuthology.orchestra.run.smithi155.stdout: Birth: - 2024-02-18T01:11:23.995 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-02-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records in 2024-02-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi155.stderr:1+0 records out 2024-02-18T01:11:24.051 INFO:teuthology.orchestra.run.smithi155.stderr:512 bytes copied, 0.000362668 s, 1.4 MB/s 2024-02-18T01:11:24.052 DEBUG:teuthology.orchestra.run.smithi155:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-02-18T01:11:24.105 INFO:tasks.cephadm:Deploying osd.0 on smithi105 with /dev/vg_nvme/lv_4... 2024-02-18T01:11:24.105 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T01:11:24.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:24 smithi155 bash[20805]: audit 2024-02-18T01:11:22.124743+0000 mgr.y (mgr.14146) 59 : audit [DBG] from='client.24109 v1:172.21.15.155:0/347113267' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi105=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:24.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:24 smithi155 bash[20805]: cephadm 2024-02-18T01:11:22.128327+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Saving service mgr spec with placement smithi105=y;smithi155=x;count:2 2024-02-18T01:11:24.307 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:24 smithi155 bash[20805]: cephadm 2024-02-18T01:11:22.179314+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Deploying daemon mgr.x on smithi155 2024-02-18T01:11:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[19734]: audit 2024-02-18T01:11:22.124743+0000 mgr.y (mgr.14146) 59 : audit [DBG] from='client.24109 v1:172.21.15.155:0/347113267' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi105=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[19734]: cephadm 2024-02-18T01:11:22.128327+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Saving service mgr spec with placement smithi105=y;smithi155=x;count:2 2024-02-18T01:11:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[19734]: cephadm 2024-02-18T01:11:22.179314+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Deploying daemon mgr.x on smithi155 2024-02-18T01:11:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[27669]: audit 2024-02-18T01:11:22.124743+0000 mgr.y (mgr.14146) 59 : audit [DBG] from='client.24109 v1:172.21.15.155:0/347113267' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi105=y;smithi155=x", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[27669]: cephadm 2024-02-18T01:11:22.128327+0000 mgr.y (mgr.14146) 60 : cephadm [INF] Saving service mgr spec with placement smithi105=y;smithi155=x;count:2 2024-02-18T01:11:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:24 smithi105 bash[27669]: cephadm 2024-02-18T01:11:22.179314+0000 mgr.y (mgr.14146) 61 : cephadm [INF] Deploying daemon mgr.x on smithi155 2024-02-18T01:11:24.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:24 smithi155 systemd[1]: Started Ceph mgr.x for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[19734]: cluster 2024-02-18T01:11:23.463428+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[19734]: audit 2024-02-18T01:11:24.618567+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[19734]: audit 2024-02-18T01:11:24.621110+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[27669]: cluster 2024-02-18T01:11:23.463428+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[27669]: audit 2024-02-18T01:11:24.618567+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:25.334 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:25 smithi105 bash[27669]: audit 2024-02-18T01:11:24.621110+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:25 smithi155 bash[20805]: cluster 2024-02-18T01:11:23.463428+0000 mgr.y (mgr.14146) 62 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:25 smithi155 bash[20805]: audit 2024-02-18T01:11:24.618567+0000 mon.a (mon.0) 203 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:25 smithi155 bash[20805]: audit 2024-02-18T01:11:24.621110+0000 mon.a (mon.0) 204 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:25.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:25 smithi155 bash[22101]: debug 2024-02-18T01:11:25.863+0000 7fa0e5929700 1 -- 172.21.15.155:0/194567917 <== mon.1 v1:172.21.15.155:6789/0 6 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (unknown 3389321392 0 0) 0x5633e55f0d00 con 0x5633e56e4c00 2024-02-18T01:11:26.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:26 smithi155 bash[22101]: debug 2024-02-18T01:11:26.023+0000 7fa0f5f24680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:11:26.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:26 smithi155 bash[22101]: debug 2024-02-18T01:11:26.123+0000 7fa0f5f24680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:11:26.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:26 smithi155 bash[22101]: debug 2024-02-18T01:11:26.555+0000 7fa0f5f24680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:11:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T01:11:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T01:11:27.438 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-02-18T01:11:27.438 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-02-18T01:11:27.439 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0175001 s, 599 MB/s 2024-02-18T01:11:27.439 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-02-18T01:11:27.453 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:27 smithi155 bash[20805]: cluster 2024-02-18T01:11:25.463859+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:27.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:27 smithi105 bash[27669]: cluster 2024-02-18T01:11:25.463859+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:27.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:27 smithi105 bash[19734]: cluster 2024-02-18T01:11:25.463859+0000 mgr.y (mgr.14146) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:27.743 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:27 smithi155 bash[22101]: debug 2024-02-18T01:11:27.447+0000 7fa0f5f24680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:11:27.743 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:27 smithi155 bash[22101]: debug 2024-02-18T01:11:27.539+0000 7fa0f5f24680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:11:28.104 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_4 2024-02-18T01:11:28.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:27 smithi155 bash[22101]: debug 2024-02-18T01:11:27.775+0000 7fa0f5f24680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:11:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:28 smithi105 bash[19734]: audit 2024-02-18T01:11:27.439521+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:28 smithi105 bash[19734]: cluster 2024-02-18T01:11:27.464243+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:28 smithi105 bash[27669]: audit 2024-02-18T01:11:27.439521+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:28 smithi105 bash[27669]: cluster 2024-02-18T01:11:27.464243+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:28 smithi155 bash[20805]: audit 2024-02-18T01:11:27.439521+0000 mon.a (mon.0) 205 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:28 smithi155 bash[20805]: cluster 2024-02-18T01:11:27.464243+0000 mgr.y (mgr.14146) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:29.155 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:28 smithi155 bash[22101]: debug 2024-02-18T01:11:28.899+0000 7fa0f5f24680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:11:29.155 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.055+0000 7fa0f5f24680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:11:29.477 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.151+0000 7fa0f5f24680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:11:29.477 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.255+0000 7fa0f5f24680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:11:29.743 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.471+0000 7fa0f5f24680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:11:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.709975+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.711453+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.713246+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.723753+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: cephadm 2024-02-18T01:11:28.728057+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.728660+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.730640+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: audit 2024-02-18T01:11:28.732408+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[20805]: cephadm 2024-02-18T01:11:28.734082+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi105 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.709975+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.711453+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.713246+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.723753+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: cephadm 2024-02-18T01:11:28.728057+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.728660+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.730640+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: audit 2024-02-18T01:11:28.732408+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.944 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[19734]: cephadm 2024-02-18T01:11:28.734082+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi105 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.709975+0000 mon.a (mon.0) 206 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.711453+0000 mon.a (mon.0) 207 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.713246+0000 mon.a (mon.0) 208 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.723753+0000 mon.a (mon.0) 209 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: cephadm 2024-02-18T01:11:28.728057+0000 mgr.y (mgr.14146) 65 : cephadm [INF] Reconfiguring mgr.y (unknown last config time)... 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.728660+0000 mon.a (mon.0) 210 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.730640+0000 mon.a (mon.0) 211 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: audit 2024-02-18T01:11:28.732408+0000 mon.a (mon.0) 212 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:29.945 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:29 smithi105 bash[27669]: cephadm 2024-02-18T01:11:28.734082+0000 mgr.y (mgr.14146) 66 : cephadm [INF] Reconfiguring daemon mgr.y on smithi105 2024-02-18T01:11:30.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.831+0000 7fa0f5f24680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:11:30.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:29 smithi155 bash[22101]: debug 2024-02-18T01:11:29.947+0000 7fa0f5f24680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:11:30.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:30 smithi105 bash[27669]: cluster 2024-02-18T01:11:29.464662+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:30.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:30 smithi105 bash[19734]: cluster 2024-02-18T01:11:29.464662+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:30 smithi155 bash[20805]: cluster 2024-02-18T01:11:29.464662+0000 mgr.y (mgr.14146) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:30.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:30 smithi155 bash[22101]: debug 2024-02-18T01:11:30.787+0000 7fa0f5f24680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:11:30.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:30 smithi155 bash[22101]: debug 2024-02-18T01:11:30.887+0000 7fa0f5f24680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:11:31.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:31 smithi155 bash[22101]: debug 2024-02-18T01:11:30.999+0000 7fa0f5f24680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:11:31.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:31 smithi155 bash[22101]: debug 2024-02-18T01:11:31.211+0000 7fa0f5f24680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:11:31.825 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:31 smithi155 bash[22101]: debug 2024-02-18T01:11:31.499+0000 7fa0f5f24680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:11:31.826 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:31 smithi155 bash[22101]: debug 2024-02-18T01:11:31.655+0000 7fa0f5f24680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:11:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.951+0000 7f5d0be7f700 1 -- 172.21.15.105:0/706348520 >> v1:172.21.15.105:6789/0 conn(0x7f5d040fbbc0 legacy=0x7f5d040fe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.951+0000 7f5d0be7f700 1 -- 172.21.15.105:0/706348520 shutdown_connections 2024-02-18T01:11:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.951+0000 7f5d0be7f700 1 -- 172.21.15.105:0/706348520 >> 172.21.15.105:0/706348520 conn(0x7f5d040f1eb0 msgr2=0x7f5d040f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:11:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.951+0000 7f5d0be7f700 1 -- 172.21.15.105:0/706348520 shutdown_connections 2024-02-18T01:11:31.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- 172.21.15.105:0/706348520 wait complete. 2024-02-18T01:11:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 Processor -- start 2024-02-18T01:11:31.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- start start 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d04108190 con 0x7f5d040fbbc0 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d041139a0 con 0x7f5d04061720 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5d04114b80 con 0x7f5d040ff920 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0941a700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f5d040fbbc0 0x7f5d0410db40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:50868/0 (socket says 172.21.15.105:50868) 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0941a700 1 -- 172.21.15.105:0/1556067249 learned_addr learned my addr 172.21.15.105:0/1556067249 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:11:31.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0a41c700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f5d040ff920 0x7f5d04111270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:33112/0 (socket says 172.21.15.105:33112) 2024-02-18T01:11:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3244336095 0 0) 0x7f5d04114b80 con 0x7f5d040ff920 2024-02-18T01:11:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf0003110 con 0x7f5d040ff920 2024-02-18T01:11:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215328287 0 0) 0x7f5d041139a0 con 0x7f5d04061720 2024-02-18T01:11:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d04114b80 con 0x7f5d04061720 2024-02-18T01:11:31.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2976464621 0 0) 0x7f5d04108190 con 0x7f5d040fbbc0 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d041139a0 con 0x7f5d040fbbc0 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2309759757 0 0) 0x7f5cf0003110 con 0x7f5d040ff920 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d04108190 con 0x7f5d040ff920 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5cfc001e80 con 0x7f5d040ff920 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2287853040 0 0) 0x7f5d04114b80 con 0x7f5d04061720 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5cf0003110 con 0x7f5d04061720 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5d000019d0 con 0x7f5d04061720 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 366064628 0 0) 0x7f5d041139a0 con 0x7f5d040fbbc0 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5d04114b80 con 0x7f5d040fbbc0 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 438711990 0 0) 0x7f5d04108190 con 0x7f5d040ff920 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 >> v1:172.21.15.155:6789/0 conn(0x7f5d04061720 legacy=0x7f5d041075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 >> v1:172.21.15.105:6789/0 conn(0x7f5d040fbbc0 legacy=0x7f5d0410db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d04115da0 con 0x7f5d040ff920 2024-02-18T01:11:31.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d04113b90 con 0x7f5d040ff920 2024-02-18T01:11:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d04114000 con 0x7f5d040ff920 2024-02-18T01:11:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f5cfc001a40 con 0x7f5d040ff920 2024-02-18T01:11:31.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.955+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5cfc005d80 con 0x7f5d040ff920 2024-02-18T01:11:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.959+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 13) v1 ==== 38639+0+0 (unknown 2967336038 0 0) 0x7f5cfc00f6c0 con 0x7f5d040ff920 2024-02-18T01:11:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.959+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(4..4 src has 1..4) v4 ==== 1012+0+0 (unknown 2904323013 0 0) 0x7f5cfc03fb40 con 0x7f5d040ff920 2024-02-18T01:11:31.963 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.959+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce8004f20 con 0x7f5d040ff920 2024-02-18T01:11:31.968 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:31.963+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5cfc0110c0 con 0x7f5d040ff920 2024-02-18T01:11:32.138 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:31 smithi155 bash[22101]: debug 2024-02-18T01:11:31.819+0000 7fa0f5f24680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:11:32.138 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[22101]: debug 2024-02-18T01:11:32.031+0000 7fa0f5f24680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:11:32.160 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:32.155+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5ce8000b90 con 0x7f5cf0033a70 2024-02-18T01:11:32.356 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:11:32.351+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f5cfc03ecb0 con 0x7f5d040ff920 2024-02-18T01:11:32.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[22101]: debug 2024-02-18T01:11:32.135+0000 7fa0f5f24680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:31.332022+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:31.334291+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:31.337258+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:31.338765+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:31.347445+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: cluster 2024-02-18T01:11:31.465103+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: cluster 2024-02-18T01:11:32.160495+0000 mon.a (mon.0) 218 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:11:32.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.163233+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.164251+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.166547+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.167539+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.168313+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.173829+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:11:32.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:32 smithi155 bash[20805]: audit 2024-02-18T01:11:32.175074+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:32.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:31.332022+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:31.334291+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:31.337258+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:31.338765+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:31.347445+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: cluster 2024-02-18T01:11:31.465103+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: cluster 2024-02-18T01:11:32.160495+0000 mon.a (mon.0) 218 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.163233+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.164251+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.166547+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.167539+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.168313+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:11:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.173829+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[19734]: audit 2024-02-18T01:11:32.175074+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:31.332022+0000 mon.a (mon.0) 213 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:31.334291+0000 mon.a (mon.0) 214 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:31.337258+0000 mon.a (mon.0) 215 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:31.338765+0000 mon.a (mon.0) 216 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:31.347445+0000 mon.a (mon.0) 217 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: cluster 2024-02-18T01:11:31.465103+0000 mgr.y (mgr.14146) 68 : cluster [DBG] pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: cluster 2024-02-18T01:11:32.160495+0000 mon.a (mon.0) 218 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.163233+0000 mon.c (mon.2) 2 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.164251+0000 mon.c (mon.2) 3 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:11:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.166547+0000 mon.c (mon.2) 4 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:11:32.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.167539+0000 mon.c (mon.2) 5 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2222918256' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:11:32.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.168313+0000 mon.a (mon.0) 219 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:11:32.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.173829+0000 mon.a (mon.0) 220 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:11:32.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:32 smithi105 bash[27669]: audit 2024-02-18T01:11:32.175074+0000 mon.a (mon.0) 221 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:33.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[19734]: audit 2024-02-18T01:11:32.161434+0000 mgr.y (mgr.14146) 69 : audit [DBG] from='client.24125 v1:172.21.15.105:0/1556067249' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:33.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[19734]: cluster 2024-02-18T01:11:32.357315+0000 mon.a (mon.0) 222 : cluster [DBG] mgrmap e14: y(active, since 96s), standbys: x 2024-02-18T01:11:33.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[19734]: audit 2024-02-18T01:11:32.357525+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:11:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[27669]: audit 2024-02-18T01:11:32.161434+0000 mgr.y (mgr.14146) 69 : audit [DBG] from='client.24125 v1:172.21.15.105:0/1556067249' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[27669]: cluster 2024-02-18T01:11:32.357315+0000 mon.a (mon.0) 222 : cluster [DBG] mgrmap e14: y(active, since 96s), standbys: x 2024-02-18T01:11:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:33 smithi105 bash[27669]: audit 2024-02-18T01:11:32.357525+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:11:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:33 smithi155 bash[20805]: audit 2024-02-18T01:11:32.161434+0000 mgr.y (mgr.14146) 69 : audit [DBG] from='client.24125 v1:172.21.15.105:0/1556067249' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:11:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:33 smithi155 bash[20805]: cluster 2024-02-18T01:11:32.357315+0000 mon.a (mon.0) 222 : cluster [DBG] mgrmap e14: y(active, since 96s), standbys: x 2024-02-18T01:11:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:33 smithi155 bash[20805]: audit 2024-02-18T01:11:32.357525+0000 mon.a (mon.0) 223 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:11:34.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:34 smithi105 bash[27669]: cluster 2024-02-18T01:11:33.465559+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:34.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:34 smithi105 bash[19734]: cluster 2024-02-18T01:11:33.465559+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:34.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:34 smithi155 bash[20805]: cluster 2024-02-18T01:11:33.465559+0000 mgr.y (mgr.14146) 70 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:36 smithi155 bash[20805]: cluster 2024-02-18T01:11:35.466005+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:36.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:36 smithi105 bash[27669]: cluster 2024-02-18T01:11:35.466005+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:36.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:36 smithi105 bash[19734]: cluster 2024-02-18T01:11:35.466005+0000 mgr.y (mgr.14146) 71 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:38 smithi105 bash[19734]: cluster 2024-02-18T01:11:37.466504+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:38 smithi105 bash[27669]: cluster 2024-02-18T01:11:37.466504+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:38 smithi155 bash[20805]: cluster 2024-02-18T01:11:37.466504+0000 mgr.y (mgr.14146) 72 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:40.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: cluster 2024-02-18T01:11:39.467025+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:40.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: audit 2024-02-18T01:11:40.288871+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.105:0/4259527971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: audit 2024-02-18T01:11:40.289265+0000 mon.a (mon.0) 224 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: audit 2024-02-18T01:11:40.294778+0000 mon.a (mon.0) 225 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]': finished 2024-02-18T01:11:40.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: cluster 2024-02-18T01:11:40.294857+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T01:11:40.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[27669]: audit 2024-02-18T01:11:40.294972+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:11:40.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: cluster 2024-02-18T01:11:39.467025+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:40.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: audit 2024-02-18T01:11:40.288871+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.105:0/4259527971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: audit 2024-02-18T01:11:40.289265+0000 mon.a (mon.0) 224 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: audit 2024-02-18T01:11:40.294778+0000 mon.a (mon.0) 225 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]': finished 2024-02-18T01:11:40.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: cluster 2024-02-18T01:11:40.294857+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T01:11:40.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:40 smithi105 bash[19734]: audit 2024-02-18T01:11:40.294972+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:11:40.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: cluster 2024-02-18T01:11:39.467025+0000 mgr.y (mgr.14146) 73 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: audit 2024-02-18T01:11:40.288871+0000 mon.c (mon.2) 6 : audit [INF] from='client.? v1:172.21.15.105:0/4259527971' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: audit 2024-02-18T01:11:40.289265+0000 mon.a (mon.0) 224 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]: dispatch 2024-02-18T01:11:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: audit 2024-02-18T01:11:40.294778+0000 mon.a (mon.0) 225 : audit [INF] from='client.24131 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62250a45-b3fb-4078-8edb-dc54138a7b9f"}]': finished 2024-02-18T01:11:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: cluster 2024-02-18T01:11:40.294857+0000 mon.a (mon.0) 226 : cluster [DBG] osdmap e5: 1 total, 0 up, 1 in 2024-02-18T01:11:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:40 smithi155 bash[20805]: audit 2024-02-18T01:11:40.294972+0000 mon.a (mon.0) 227 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:11:41.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:41 smithi105 bash[19734]: audit 2024-02-18T01:11:40.877446+0000 mon.a (mon.0) 228 : audit [DBG] from='client.? v1:172.21.15.105:0/3200177761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:11:41.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:41 smithi105 bash[27669]: audit 2024-02-18T01:11:40.877446+0000 mon.a (mon.0) 228 : audit [DBG] from='client.? v1:172.21.15.105:0/3200177761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:11:41.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:41 smithi155 bash[20805]: audit 2024-02-18T01:11:40.877446+0000 mon.a (mon.0) 228 : audit [DBG] from='client.? v1:172.21.15.105:0/3200177761' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:11:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:42 smithi105 bash[27669]: cluster 2024-02-18T01:11:41.467506+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:42 smithi105 bash[19734]: cluster 2024-02-18T01:11:41.467506+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:42 smithi155 bash[20805]: cluster 2024-02-18T01:11:41.467506+0000 mgr.y (mgr.14146) 74 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:44 smithi105 bash[19734]: cluster 2024-02-18T01:11:43.467942+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:44.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:44 smithi105 bash[27669]: cluster 2024-02-18T01:11:43.467942+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:44 smithi155 bash[20805]: cluster 2024-02-18T01:11:43.467942+0000 mgr.y (mgr.14146) 75 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:46.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:46 smithi105 bash[19734]: cluster 2024-02-18T01:11:45.468409+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:46 smithi105 bash[27669]: cluster 2024-02-18T01:11:45.468409+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:46 smithi155 bash[20805]: cluster 2024-02-18T01:11:45.468409+0000 mgr.y (mgr.14146) 76 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:48.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:48 smithi105 bash[19734]: cluster 2024-02-18T01:11:47.468836+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:48 smithi105 bash[27669]: cluster 2024-02-18T01:11:47.468836+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:48 smithi155 bash[20805]: cluster 2024-02-18T01:11:47.468836+0000 mgr.y (mgr.14146) 77 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:50.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:50 smithi105 bash[19734]: cluster 2024-02-18T01:11:49.469295+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:50.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:50 smithi105 bash[27669]: cluster 2024-02-18T01:11:49.469295+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:50 smithi155 bash[20805]: cluster 2024-02-18T01:11:49.469295+0000 mgr.y (mgr.14146) 78 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:52.979 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[27669]: cluster 2024-02-18T01:11:51.469683+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:52.979 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[27669]: audit 2024-02-18T01:11:51.993192+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T01:11:52.980 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[27669]: audit 2024-02-18T01:11:51.994779+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:52.980 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[27669]: cephadm 2024-02-18T01:11:51.996138+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Deploying daemon osd.0 on smithi105 2024-02-18T01:11:52.980 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[19734]: cluster 2024-02-18T01:11:51.469683+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:52.981 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[19734]: audit 2024-02-18T01:11:51.993192+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T01:11:52.981 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[19734]: audit 2024-02-18T01:11:51.994779+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:52.981 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:52 smithi105 bash[19734]: cephadm 2024-02-18T01:11:51.996138+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Deploying daemon osd.0 on smithi105 2024-02-18T01:11:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:52 smithi155 bash[20805]: cluster 2024-02-18T01:11:51.469683+0000 mgr.y (mgr.14146) 79 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:52 smithi155 bash[20805]: audit 2024-02-18T01:11:51.993192+0000 mon.a (mon.0) 229 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-02-18T01:11:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:52 smithi155 bash[20805]: audit 2024-02-18T01:11:51.994779+0000 mon.a (mon.0) 230 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:11:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:52 smithi155 bash[20805]: cephadm 2024-02-18T01:11:51.996138+0000 mgr.y (mgr.14146) 80 : cephadm [INF] Deploying daemon osd.0 on smithi105 2024-02-18T01:11:54.958 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:54 smithi105 bash[27669]: cluster 2024-02-18T01:11:53.470242+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:54.958 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:54 smithi105 bash[19734]: cluster 2024-02-18T01:11:53.470242+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:54 smithi155 bash[20805]: cluster 2024-02-18T01:11:53.470242+0000 mgr.y (mgr.14146) 81 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[19734]: cluster 2024-02-18T01:11:55.470710+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[19734]: audit 2024-02-18T01:11:55.816885+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[19734]: audit 2024-02-18T01:11:55.830400+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:57.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[27669]: cluster 2024-02-18T01:11:55.470710+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[27669]: audit 2024-02-18T01:11:55.816885+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:56 smithi105 bash[27669]: audit 2024-02-18T01:11:55.830400+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:56 smithi155 bash[20805]: cluster 2024-02-18T01:11:55.470710+0000 mgr.y (mgr.14146) 82 : cluster [DBG] pgmap v55: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:56 smithi155 bash[20805]: audit 2024-02-18T01:11:55.816885+0000 mon.a (mon.0) 231 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:11:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:56 smithi155 bash[20805]: audit 2024-02-18T01:11:55.830400+0000 mon.a (mon.0) 232 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:11:59.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:11:58 smithi105 bash[19734]: cluster 2024-02-18T01:11:57.471035+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:59.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:11:58 smithi105 bash[27669]: cluster 2024-02-18T01:11:57.471035+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:11:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:11:58 smithi155 bash[20805]: cluster 2024-02-18T01:11:57.471035+0000 mgr.y (mgr.14146) 83 : cluster [DBG] pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:01.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:00 smithi105 bash[19734]: cluster 2024-02-18T01:11:59.471463+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:01.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:00 smithi105 bash[27669]: cluster 2024-02-18T01:11:59.471463+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:01.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:00 smithi155 bash[20805]: cluster 2024-02-18T01:11:59.471463+0000 mgr.y (mgr.14146) 84 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[27669]: audit 2024-02-18T01:12:01.206076+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[27669]: audit 2024-02-18T01:12:01.207482+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[27669]: audit 2024-02-18T01:12:01.208935+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[27669]: audit 2024-02-18T01:12:01.217597+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[27669]: cluster 2024-02-18T01:12:01.471841+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[19734]: audit 2024-02-18T01:12:01.206076+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[19734]: audit 2024-02-18T01:12:01.207482+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[19734]: audit 2024-02-18T01:12:01.208935+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[19734]: audit 2024-02-18T01:12:01.217597+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:02 smithi105 bash[19734]: cluster 2024-02-18T01:12:01.471841+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:02.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:02 smithi155 bash[20805]: audit 2024-02-18T01:12:01.206076+0000 mon.a (mon.0) 233 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:02 smithi155 bash[20805]: audit 2024-02-18T01:12:01.207482+0000 mon.a (mon.0) 234 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:02 smithi155 bash[20805]: audit 2024-02-18T01:12:01.208935+0000 mon.a (mon.0) 235 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:02 smithi155 bash[20805]: audit 2024-02-18T01:12:01.217597+0000 mon.a (mon.0) 236 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:02 smithi155 bash[20805]: cluster 2024-02-18T01:12:01.471841+0000 mgr.y (mgr.14146) 85 : cluster [DBG] pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:03 smithi105 bash[27669]: audit 2024-02-18T01:12:02.434865+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:03 smithi105 bash[27669]: audit 2024-02-18T01:12:02.435199+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:03 smithi105 bash[19734]: audit 2024-02-18T01:12:02.434865+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:03 smithi105 bash[19734]: audit 2024-02-18T01:12:02.435199+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:03 smithi155 bash[20805]: audit 2024-02-18T01:12:02.434865+0000 mon.c (mon.2) 7 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:03 smithi155 bash[20805]: audit 2024-02-18T01:12:02.435199+0000 mon.a (mon.0) 237 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-02-18T01:12:04.443 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.439+0000 7f5cfaffd700 1 -- 172.21.15.105:0/1556067249 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 522174137) 0x7f5ce8000b90 con 0x7f5cf0033a70 2024-02-18T01:12:04.444 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 0 on host 'smithi105' 2024-02-18T01:12:04.447 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 >> v1:172.21.15.105:6800/3087430119 conn(0x7f5cf0033a70 legacy=0x7f5cf0035f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 >> v1:172.21.15.105:6790/0 conn(0x7f5d040ff920 legacy=0x7f5d04111270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 shutdown_connections 2024-02-18T01:12:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 >> 172.21.15.105:0/1556067249 conn(0x7f5d040f1eb0 msgr2=0x7f5d040f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:12:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 shutdown_connections 2024-02-18T01:12:04.448 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:04.443+0000 7f5d0be7f700 1 -- 172.21.15.105:0/1556067249 wait complete. 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: audit 2024-02-18T01:12:03.227254+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: cluster 2024-02-18T01:12:03.227361+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: audit 2024-02-18T01:12:03.227530+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: audit 2024-02-18T01:12:03.230347+0000 mon.c (mon.2) 8 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: audit 2024-02-18T01:12:03.230873+0000 mon.a (mon.0) 241 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[19734]: cluster 2024-02-18T01:12:03.472292+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:04.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: audit 2024-02-18T01:12:03.227254+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T01:12:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: cluster 2024-02-18T01:12:03.227361+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T01:12:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: audit 2024-02-18T01:12:03.227530+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: audit 2024-02-18T01:12:03.230347+0000 mon.c (mon.2) 8 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: audit 2024-02-18T01:12:03.230873+0000 mon.a (mon.0) 241 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:04 smithi105 bash[27669]: cluster 2024-02-18T01:12:03.472292+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: audit 2024-02-18T01:12:03.227254+0000 mon.a (mon.0) 238 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-02-18T01:12:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: cluster 2024-02-18T01:12:03.227361+0000 mon.a (mon.0) 239 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2024-02-18T01:12:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: audit 2024-02-18T01:12:03.227530+0000 mon.a (mon.0) 240 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: audit 2024-02-18T01:12:03.230347+0000 mon.c (mon.2) 8 : audit [INF] from='osd.0 v1:172.21.15.105:6801/3115838352' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: audit 2024-02-18T01:12:03.230873+0000 mon.a (mon.0) 241 : audit [INF] from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:04 smithi155 bash[20805]: cluster 2024-02-18T01:12:03.472292+0000 mgr.y (mgr.14146) 86 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:05.066 DEBUG:teuthology.orchestra.run.smithi105:osd.0> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.0.service 2024-02-18T01:12:05.068 INFO:tasks.cephadm:Deploying osd.1 on smithi105 with /dev/vg_nvme/lv_3... 2024-02-18T01:12:05.068 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T01:12:05.095 INFO:journalctl@ceph.osd.0.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: audit 2024-02-18T01:12:04.231618+0000 mon.a (mon.0) 242 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: cluster 2024-02-18T01:12:04.231837+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: audit 2024-02-18T01:12:04.232724+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: audit 2024-02-18T01:12:04.239577+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: audit 2024-02-18T01:12:04.442419+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[19734]: audit 2024-02-18T01:12:04.442982+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: audit 2024-02-18T01:12:04.231618+0000 mon.a (mon.0) 242 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: cluster 2024-02-18T01:12:04.231837+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: audit 2024-02-18T01:12:04.232724+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: audit 2024-02-18T01:12:04.239577+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: audit 2024-02-18T01:12:04.442419+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:05 smithi105 bash[27669]: audit 2024-02-18T01:12:04.442982+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: audit 2024-02-18T01:12:04.231618+0000 mon.a (mon.0) 242 : audit [INF] from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: cluster 2024-02-18T01:12:04.231837+0000 mon.a (mon.0) 243 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2024-02-18T01:12:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: audit 2024-02-18T01:12:04.232724+0000 mon.a (mon.0) 244 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: audit 2024-02-18T01:12:04.239577+0000 mon.a (mon.0) 245 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: audit 2024-02-18T01:12:04.442419+0000 mon.a (mon.0) 246 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:05 smithi155 bash[20805]: audit 2024-02-18T01:12:04.442982+0000 mon.a (mon.0) 247 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: cluster 2024-02-18T01:12:03.400732+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: cluster 2024-02-18T01:12:03.400866+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: cluster 2024-02-18T01:12:05.241581+0000 mon.a (mon.0) 248 : cluster [INF] osd.0 v1:172.21.15.105:6801/3115838352 boot 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: cluster 2024-02-18T01:12:05.241645+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: audit 2024-02-18T01:12:05.242265+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:06.455 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[27669]: cluster 2024-02-18T01:12:05.472812+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: cluster 2024-02-18T01:12:03.400732+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: cluster 2024-02-18T01:12:03.400866+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: cluster 2024-02-18T01:12:05.241581+0000 mon.a (mon.0) 248 : cluster [INF] osd.0 v1:172.21.15.105:6801/3115838352 boot 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: cluster 2024-02-18T01:12:05.241645+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: audit 2024-02-18T01:12:05.242265+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:06.456 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:06 smithi105 bash[19734]: cluster 2024-02-18T01:12:05.472812+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:06.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: cluster 2024-02-18T01:12:03.400732+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:06.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: cluster 2024-02-18T01:12:03.400866+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:06.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: cluster 2024-02-18T01:12:05.241581+0000 mon.a (mon.0) 248 : cluster [INF] osd.0 v1:172.21.15.105:6801/3115838352 boot 2024-02-18T01:12:06.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: cluster 2024-02-18T01:12:05.241645+0000 mon.a (mon.0) 249 : cluster [DBG] osdmap e8: 1 total, 1 up, 1 in 2024-02-18T01:12:06.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: audit 2024-02-18T01:12:05.242265+0000 mon.a (mon.0) 250 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:12:06.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:06 smithi155 bash[20805]: cluster 2024-02-18T01:12:05.472812+0000 mgr.y (mgr.14146) 87 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-02-18T01:12:07.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:07 smithi105 bash[19734]: cluster 2024-02-18T01:12:06.255111+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T01:12:07.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:07 smithi105 bash[27669]: cluster 2024-02-18T01:12:06.255111+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T01:12:07.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:07 smithi155 bash[20805]: cluster 2024-02-18T01:12:06.255111+0000 mon.a (mon.0) 251 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.01564 s, 670 MB/s 2024-02-18T01:12:08.502 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-02-18T01:12:08.582 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:08 smithi105 bash[19734]: cluster 2024-02-18T01:12:07.473367+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:08.582 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:08 smithi105 bash[27669]: cluster 2024-02-18T01:12:07.473367+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:08.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:08 smithi155 bash[20805]: cluster 2024-02-18T01:12:07.473367+0000 mgr.y (mgr.14146) 88 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:09.135 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_3 2024-02-18T01:12:10.864 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:10 smithi105 bash[19734]: cluster 2024-02-18T01:12:09.473951+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:10.864 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:10 smithi105 bash[27669]: cluster 2024-02-18T01:12:09.473951+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:10.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:10 smithi155 bash[20805]: cluster 2024-02-18T01:12:09.473951+0000 mgr.y (mgr.14146) 89 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:12.860 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:12 smithi105 bash[19734]: cluster 2024-02-18T01:12:11.474449+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:12.860 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:12 smithi105 bash[27669]: cluster 2024-02-18T01:12:11.474449+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:12.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:12 smithi155 bash[20805]: cluster 2024-02-18T01:12:11.474449+0000 mgr.y (mgr.14146) 90 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.163+0000 7f8579662700 1 -- 172.21.15.105:0/3603296989 >> v1:172.21.15.105:6790/0 conn(0x7f85740f61c0 legacy=0x7f85740f65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/3603296989 shutdown_connections 2024-02-18T01:12:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/3603296989 >> 172.21.15.105:0/3603296989 conn(0x7f85740f11c0 msgr2=0x7f85740f35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:12:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/3603296989 shutdown_connections 2024-02-18T01:12:13.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/3603296989 wait complete. 2024-02-18T01:12:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 Processor -- start 2024-02-18T01:12:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- start start 2024-02-18T01:12:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f85741128c0 con 0x7f85740fdfa0 2024-02-18T01:12:13.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8574113a60 con 0x7f85740f61c0 2024-02-18T01:12:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8574114c00 con 0x7f85740fa340 2024-02-18T01:12:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f85727fc700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f85740fa340 0x7f857410da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:51006/0 (socket says 172.21.15.105:51006) 2024-02-18T01:12:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f85727fc700 1 -- 172.21.15.105:0/888218447 learned_addr learned my addr 172.21.15.105:0/888218447 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:12:13.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247153356 0 0) 0x7f8574114c00 con 0x7f85740fa340 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f855c0033f0 con 0x7f85740fa340 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3512704257 0 0) 0x7f85741128c0 con 0x7f85740fdfa0 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8574114c00 con 0x7f85740fdfa0 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244160451 0 0) 0x7f8574113a60 con 0x7f85740f61c0 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85741128c0 con 0x7f85740f61c0 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3462673315 0 0) 0x7f855c0033f0 con 0x7f85740fa340 2024-02-18T01:12:13.172 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8574113a60 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f85680019d0 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 437822404 0 0) 0x7f8574114c00 con 0x7f85740fdfa0 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f855c0033f0 con 0x7f85740fdfa0 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f856c001bc0 con 0x7f85740fdfa0 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2688709699 0 0) 0x7f8574113a60 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 >> v1:172.21.15.155:6789/0 conn(0x7f85740f61c0 legacy=0x7f857410a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 >> v1:172.21.15.105:6789/0 conn(0x7f85740fdfa0 legacy=0x7f85741111a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8574115da0 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8574112ab0 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8574112f20 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f8568002400 con 0x7f85740fa340 2024-02-18T01:12:13.173 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.167+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8568005a90 con 0x7f85740fa340 2024-02-18T01:12:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.171+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8560004f20 con 0x7f85740fa340 2024-02-18T01:12:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.171+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f8568018900 con 0x7f85740fa340 2024-02-18T01:12:13.174 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.171+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(9..9 src has 1..9) v4 ==== 1484+0+0 (unknown 191821502 0 0) 0x7f85680771c0 con 0x7f85740fa340 2024-02-18T01:12:13.180 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.175+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f85680485f0 con 0x7f85740fa340 2024-02-18T01:12:13.375 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:13.371+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8560000b90 con 0x7f855c062270 2024-02-18T01:12:13.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[19734]: audit 2024-02-18T01:12:13.382561+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:13.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[19734]: audit 2024-02-18T01:12:13.385801+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:13.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[19734]: audit 2024-02-18T01:12:13.386641+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:13.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[27669]: audit 2024-02-18T01:12:13.382561+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[27669]: audit 2024-02-18T01:12:13.385801+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:13 smithi105 bash[27669]: audit 2024-02-18T01:12:13.386641+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:13 smithi155 bash[20805]: audit 2024-02-18T01:12:13.382561+0000 mon.a (mon.0) 252 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:13 smithi155 bash[20805]: audit 2024-02-18T01:12:13.385801+0000 mon.a (mon.0) 253 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:13 smithi155 bash[20805]: audit 2024-02-18T01:12:13.386641+0000 mon.a (mon.0) 254 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[19734]: audit 2024-02-18T01:12:13.375933+0000 mgr.y (mgr.14146) 91 : audit [DBG] from='client.24152 v1:172.21.15.105:0/888218447' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[19734]: cluster 2024-02-18T01:12:13.474825+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[19734]: audit 2024-02-18T01:12:14.504579+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:14.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[27669]: audit 2024-02-18T01:12:13.375933+0000 mgr.y (mgr.14146) 91 : audit [DBG] from='client.24152 v1:172.21.15.105:0/888218447' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[27669]: cluster 2024-02-18T01:12:13.474825+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:14 smithi105 bash[27669]: audit 2024-02-18T01:12:14.504579+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:14 smithi155 bash[20805]: audit 2024-02-18T01:12:13.375933+0000 mgr.y (mgr.14146) 91 : audit [DBG] from='client.24152 v1:172.21.15.105:0/888218447' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:14 smithi155 bash[20805]: cluster 2024-02-18T01:12:13.474825+0000 mgr.y (mgr.14146) 92 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:14 smithi155 bash[20805]: audit 2024-02-18T01:12:14.504579+0000 mon.a (mon.0) 255 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: cephadm 2024-02-18T01:12:14.496294+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: audit 2024-02-18T01:12:14.788469+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: cephadm 2024-02-18T01:12:14.789626+0000 mgr.y (mgr.14146) 94 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 16294M 2024-02-18T01:12:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: audit 2024-02-18T01:12:14.796491+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: audit 2024-02-18T01:12:14.797910+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: audit 2024-02-18T01:12:14.799179+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[19734]: audit 2024-02-18T01:12:14.804861+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: cephadm 2024-02-18T01:12:14.496294+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: audit 2024-02-18T01:12:14.788469+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: cephadm 2024-02-18T01:12:14.789626+0000 mgr.y (mgr.14146) 94 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 16294M 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: audit 2024-02-18T01:12:14.796491+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: audit 2024-02-18T01:12:14.797910+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: audit 2024-02-18T01:12:14.799179+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:15 smithi105 bash[27669]: audit 2024-02-18T01:12:14.804861+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: cephadm 2024-02-18T01:12:14.496294+0000 mgr.y (mgr.14146) 93 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: audit 2024-02-18T01:12:14.788469+0000 mon.a (mon.0) 256 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: cephadm 2024-02-18T01:12:14.789626+0000 mgr.y (mgr.14146) 94 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 16294M 2024-02-18T01:12:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: audit 2024-02-18T01:12:14.796491+0000 mon.a (mon.0) 257 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: audit 2024-02-18T01:12:14.797910+0000 mon.a (mon.0) 258 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: audit 2024-02-18T01:12:14.799179+0000 mon.a (mon.0) 259 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:15 smithi155 bash[20805]: audit 2024-02-18T01:12:14.804861+0000 mon.a (mon.0) 260 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:17.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:16 smithi105 bash[19734]: cluster 2024-02-18T01:12:15.475342+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:17.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:16 smithi105 bash[27669]: cluster 2024-02-18T01:12:15.475342+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:16 smithi155 bash[20805]: cluster 2024-02-18T01:12:15.475342+0000 mgr.y (mgr.14146) 95 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:19.113 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:18 smithi105 bash[19734]: cluster 2024-02-18T01:12:17.475843+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:19.114 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:18 smithi105 bash[27669]: cluster 2024-02-18T01:12:17.475843+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:18 smithi155 bash[20805]: cluster 2024-02-18T01:12:17.475843+0000 mgr.y (mgr.14146) 96 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:20.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:20 smithi105 bash[19734]: cluster 2024-02-18T01:12:19.476277+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:20.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:20 smithi105 bash[27669]: cluster 2024-02-18T01:12:19.476277+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:20 smithi155 bash[20805]: cluster 2024-02-18T01:12:19.476277+0000 mgr.y (mgr.14146) 97 : cluster [DBG] pgmap v71: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[19734]: audit 2024-02-18T01:12:21.446799+0000 mon.c (mon.2) 9 : audit [INF] from='client.? v1:172.21.15.105:0/461559296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[19734]: audit 2024-02-18T01:12:21.447432+0000 mon.a (mon.0) 261 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[19734]: audit 2024-02-18T01:12:21.459781+0000 mon.a (mon.0) 262 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]': finished 2024-02-18T01:12:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[19734]: cluster 2024-02-18T01:12:21.459971+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T01:12:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[19734]: audit 2024-02-18T01:12:21.460126+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[27669]: audit 2024-02-18T01:12:21.446799+0000 mon.c (mon.2) 9 : audit [INF] from='client.? v1:172.21.15.105:0/461559296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[27669]: audit 2024-02-18T01:12:21.447432+0000 mon.a (mon.0) 261 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[27669]: audit 2024-02-18T01:12:21.459781+0000 mon.a (mon.0) 262 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]': finished 2024-02-18T01:12:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[27669]: cluster 2024-02-18T01:12:21.459971+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T01:12:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:21 smithi105 bash[27669]: audit 2024-02-18T01:12:21.460126+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:21 smithi155 bash[20805]: audit 2024-02-18T01:12:21.446799+0000 mon.c (mon.2) 9 : audit [INF] from='client.? v1:172.21.15.105:0/461559296' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:21 smithi155 bash[20805]: audit 2024-02-18T01:12:21.447432+0000 mon.a (mon.0) 261 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]: dispatch 2024-02-18T01:12:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:21 smithi155 bash[20805]: audit 2024-02-18T01:12:21.459781+0000 mon.a (mon.0) 262 : audit [INF] from='client.24158 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "59487fa0-277f-4175-b49b-f50ee2645931"}]': finished 2024-02-18T01:12:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:21 smithi155 bash[20805]: cluster 2024-02-18T01:12:21.459971+0000 mon.a (mon.0) 263 : cluster [DBG] osdmap e10: 2 total, 1 up, 2 in 2024-02-18T01:12:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:21 smithi155 bash[20805]: audit 2024-02-18T01:12:21.460126+0000 mon.a (mon.0) 264 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:22 smithi105 bash[19734]: cluster 2024-02-18T01:12:21.476623+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:22 smithi105 bash[19734]: audit 2024-02-18T01:12:22.052627+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? v1:172.21.15.105:0/1826578173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:12:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:22 smithi105 bash[27669]: cluster 2024-02-18T01:12:21.476623+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:22 smithi105 bash[27669]: audit 2024-02-18T01:12:22.052627+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? v1:172.21.15.105:0/1826578173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:12:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:22 smithi155 bash[20805]: cluster 2024-02-18T01:12:21.476623+0000 mgr.y (mgr.14146) 98 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:22 smithi155 bash[20805]: audit 2024-02-18T01:12:22.052627+0000 mon.c (mon.2) 10 : audit [DBG] from='client.? v1:172.21.15.105:0/1826578173' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:12:25.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:24 smithi105 bash[19734]: cluster 2024-02-18T01:12:23.477098+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:24 smithi105 bash[27669]: cluster 2024-02-18T01:12:23.477098+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:24 smithi155 bash[20805]: cluster 2024-02-18T01:12:23.477098+0000 mgr.y (mgr.14146) 99 : cluster [DBG] pgmap v74: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:27.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:26 smithi105 bash[19734]: cluster 2024-02-18T01:12:25.477610+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:26 smithi105 bash[27669]: cluster 2024-02-18T01:12:25.477610+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:26 smithi155 bash[20805]: cluster 2024-02-18T01:12:25.477610+0000 mgr.y (mgr.14146) 100 : cluster [DBG] pgmap v75: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:29.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:28 smithi105 bash[19734]: cluster 2024-02-18T01:12:27.478047+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:28 smithi105 bash[27669]: cluster 2024-02-18T01:12:27.478047+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:28 smithi155 bash[20805]: cluster 2024-02-18T01:12:27.478047+0000 mgr.y (mgr.14146) 101 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:31.023 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:30 smithi105 bash[19734]: cluster 2024-02-18T01:12:29.478583+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:31.023 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:30 smithi105 bash[27669]: cluster 2024-02-18T01:12:29.478583+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:30 smithi155 bash[20805]: cluster 2024-02-18T01:12:29.478583+0000 mgr.y (mgr.14146) 102 : cluster [DBG] pgmap v77: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:32 smithi105 bash[19734]: cluster 2024-02-18T01:12:31.479003+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:32 smithi105 bash[27669]: cluster 2024-02-18T01:12:31.479003+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:32 smithi155 bash[20805]: cluster 2024-02-18T01:12:31.479003+0000 mgr.y (mgr.14146) 103 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:34.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:33 smithi105 bash[27669]: audit 2024-02-18T01:12:33.277086+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T01:12:34.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:33 smithi105 bash[27669]: audit 2024-02-18T01:12:33.278227+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:34.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:33 smithi105 bash[19734]: audit 2024-02-18T01:12:33.277086+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T01:12:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:33 smithi105 bash[19734]: audit 2024-02-18T01:12:33.278227+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:34.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:33 smithi155 bash[20805]: audit 2024-02-18T01:12:33.277086+0000 mon.a (mon.0) 265 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-02-18T01:12:34.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:33 smithi155 bash[20805]: audit 2024-02-18T01:12:33.278227+0000 mon.a (mon.0) 266 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:35.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:34 smithi105 bash[19734]: cephadm 2024-02-18T01:12:33.279178+0000 mgr.y (mgr.14146) 104 : cephadm [INF] Deploying daemon osd.1 on smithi105 2024-02-18T01:12:35.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:34 smithi105 bash[19734]: cluster 2024-02-18T01:12:33.479490+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:35.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:34 smithi105 bash[27669]: cephadm 2024-02-18T01:12:33.279178+0000 mgr.y (mgr.14146) 104 : cephadm [INF] Deploying daemon osd.1 on smithi105 2024-02-18T01:12:35.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:34 smithi105 bash[27669]: cluster 2024-02-18T01:12:33.479490+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:34 smithi155 bash[20805]: cephadm 2024-02-18T01:12:33.279178+0000 mgr.y (mgr.14146) 104 : cephadm [INF] Deploying daemon osd.1 on smithi105 2024-02-18T01:12:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:34 smithi155 bash[20805]: cluster 2024-02-18T01:12:33.479490+0000 mgr.y (mgr.14146) 105 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:37.187 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:36 smithi105 bash[19734]: cluster 2024-02-18T01:12:35.480003+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:37.188 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:36 smithi105 bash[27669]: cluster 2024-02-18T01:12:35.480003+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:36 smithi155 bash[20805]: cluster 2024-02-18T01:12:35.480003+0000 mgr.y (mgr.14146) 106 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:38.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[27669]: audit 2024-02-18T01:12:37.224249+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:38.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[27669]: audit 2024-02-18T01:12:37.228738+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:38.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[27669]: cluster 2024-02-18T01:12:37.480337+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:38.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[19734]: audit 2024-02-18T01:12:37.224249+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:38.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[19734]: audit 2024-02-18T01:12:37.228738+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:38.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:38 smithi105 bash[19734]: cluster 2024-02-18T01:12:37.480337+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:38 smithi155 bash[20805]: audit 2024-02-18T01:12:37.224249+0000 mon.a (mon.0) 267 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:38 smithi155 bash[20805]: audit 2024-02-18T01:12:37.228738+0000 mon.a (mon.0) 268 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:38 smithi155 bash[20805]: cluster 2024-02-18T01:12:37.480337+0000 mgr.y (mgr.14146) 107 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:40.884 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:40 smithi105 bash[27669]: cluster 2024-02-18T01:12:39.480778+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:40.884 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:40 smithi105 bash[19734]: cluster 2024-02-18T01:12:39.480778+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:40 smithi155 bash[20805]: cluster 2024-02-18T01:12:39.480778+0000 mgr.y (mgr.14146) 108 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:42.955 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:42 smithi105 bash[19734]: cluster 2024-02-18T01:12:41.481165+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:42.955 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:42 smithi105 bash[27669]: cluster 2024-02-18T01:12:41.481165+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:42 smithi155 bash[20805]: cluster 2024-02-18T01:12:41.481165+0000 mgr.y (mgr.14146) 109 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:43.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[19734]: audit 2024-02-18T01:12:42.586126+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:43.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[19734]: audit 2024-02-18T01:12:42.587508+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:43.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[19734]: audit 2024-02-18T01:12:42.589110+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:43.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[19734]: audit 2024-02-18T01:12:42.596458+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:43.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[27669]: audit 2024-02-18T01:12:42.586126+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:43.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[27669]: audit 2024-02-18T01:12:42.587508+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:43.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[27669]: audit 2024-02-18T01:12:42.589110+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:43.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:43 smithi105 bash[27669]: audit 2024-02-18T01:12:42.596458+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:43.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:43 smithi155 bash[20805]: audit 2024-02-18T01:12:42.586126+0000 mon.a (mon.0) 269 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:43.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:43 smithi155 bash[20805]: audit 2024-02-18T01:12:42.587508+0000 mon.a (mon.0) 270 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:43.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:43 smithi155 bash[20805]: audit 2024-02-18T01:12:42.589110+0000 mon.a (mon.0) 271 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:43.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:43 smithi155 bash[20805]: audit 2024-02-18T01:12:42.596458+0000 mon.a (mon.0) 272 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:44 smithi105 bash[19734]: cluster 2024-02-18T01:12:43.481642+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:44 smithi105 bash[19734]: audit 2024-02-18T01:12:43.833208+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T01:12:44.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:44 smithi105 bash[27669]: cluster 2024-02-18T01:12:43.481642+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:44.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:44 smithi105 bash[27669]: audit 2024-02-18T01:12:43.833208+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T01:12:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:44 smithi155 bash[20805]: cluster 2024-02-18T01:12:43.481642+0000 mgr.y (mgr.14146) 110 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:44 smithi155 bash[20805]: audit 2024-02-18T01:12:43.833208+0000 mon.a (mon.0) 273 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-02-18T01:12:45.782 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.774+0000 7f855bfff700 1 -- 172.21.15.105:0/888218447 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1124749325) 0x7f8560000b90 con 0x7f855c062270 2024-02-18T01:12:45.782 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 1 on host 'smithi105' 2024-02-18T01:12:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 >> v1:172.21.15.105:6800/3087430119 conn(0x7f855c062270 legacy=0x7f855c064730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 >> v1:172.21.15.105:6790/0 conn(0x7f85740fa340 legacy=0x7f857410da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:45.784 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 shutdown_connections 2024-02-18T01:12:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 >> 172.21.15.105:0/888218447 conn(0x7f85740f11c0 msgr2=0x7f8574100460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:12:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 shutdown_connections 2024-02-18T01:12:45.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:45.778+0000 7f8579662700 1 -- 172.21.15.105:0/888218447 wait complete. 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[19734]: audit 2024-02-18T01:12:44.609651+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[19734]: cluster 2024-02-18T01:12:44.609810+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[19734]: audit 2024-02-18T01:12:44.609986+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[19734]: audit 2024-02-18T01:12:44.610642+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[27669]: audit 2024-02-18T01:12:44.609651+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[27669]: cluster 2024-02-18T01:12:44.609810+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T01:12:45.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[27669]: audit 2024-02-18T01:12:44.609986+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:45 smithi105 bash[27669]: audit 2024-02-18T01:12:44.610642+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:45 smithi155 bash[20805]: audit 2024-02-18T01:12:44.609651+0000 mon.a (mon.0) 274 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-02-18T01:12:45.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:45 smithi155 bash[20805]: cluster 2024-02-18T01:12:44.609810+0000 mon.a (mon.0) 275 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2024-02-18T01:12:45.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:45 smithi155 bash[20805]: audit 2024-02-18T01:12:44.609986+0000 mon.a (mon.0) 276 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:45.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:45 smithi155 bash[20805]: audit 2024-02-18T01:12:44.610642+0000 mon.a (mon.0) 277 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:12:46.369 DEBUG:teuthology.orchestra.run.smithi105:osd.1> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.1.service 2024-02-18T01:12:46.371 INFO:tasks.cephadm:Deploying osd.2 on smithi105 with /dev/vg_nvme/lv_2... 2024-02-18T01:12:46.371 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T01:12:46.626 INFO:journalctl@ceph.osd.1.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: cluster 2024-02-18T01:12:45.482114+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: audit 2024-02-18T01:12:45.612883+0000 mon.a (mon.0) 278 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: cluster 2024-02-18T01:12:45.613084+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: audit 2024-02-18T01:12:45.613927+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: audit 2024-02-18T01:12:45.622465+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: audit 2024-02-18T01:12:45.772542+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[27669]: audit 2024-02-18T01:12:45.776587+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: cluster 2024-02-18T01:12:45.482114+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: audit 2024-02-18T01:12:45.612883+0000 mon.a (mon.0) 278 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: cluster 2024-02-18T01:12:45.613084+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: audit 2024-02-18T01:12:45.613927+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: audit 2024-02-18T01:12:45.622465+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: audit 2024-02-18T01:12:45.772542+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:46 smithi105 bash[19734]: audit 2024-02-18T01:12:45.776587+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: cluster 2024-02-18T01:12:45.482114+0000 mgr.y (mgr.14146) 111 : cluster [DBG] pgmap v86: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: audit 2024-02-18T01:12:45.612883+0000 mon.a (mon.0) 278 : audit [INF] from='osd.1 v1:172.21.15.105:6805/1532722979' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:12:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: cluster 2024-02-18T01:12:45.613084+0000 mon.a (mon.0) 279 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2024-02-18T01:12:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: audit 2024-02-18T01:12:45.613927+0000 mon.a (mon.0) 280 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: audit 2024-02-18T01:12:45.622465+0000 mon.a (mon.0) 281 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: audit 2024-02-18T01:12:45.772542+0000 mon.a (mon.0) 282 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:12:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:46 smithi155 bash[20805]: audit 2024-02-18T01:12:45.776587+0000 mon.a (mon.0) 283 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[27669]: cluster 2024-02-18T01:12:44.855350+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[27669]: cluster 2024-02-18T01:12:44.855447+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[27669]: cluster 2024-02-18T01:12:46.621188+0000 mon.a (mon.0) 284 : cluster [INF] osd.1 v1:172.21.15.105:6805/1532722979 boot 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[27669]: cluster 2024-02-18T01:12:46.621255+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[27669]: audit 2024-02-18T01:12:46.622690+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[19734]: cluster 2024-02-18T01:12:44.855350+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:47.797 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[19734]: cluster 2024-02-18T01:12:44.855447+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:47.798 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[19734]: cluster 2024-02-18T01:12:46.621188+0000 mon.a (mon.0) 284 : cluster [INF] osd.1 v1:172.21.15.105:6805/1532722979 boot 2024-02-18T01:12:47.798 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[19734]: cluster 2024-02-18T01:12:46.621255+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T01:12:47.798 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:47 smithi105 bash[19734]: audit 2024-02-18T01:12:46.622690+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:47 smithi155 bash[20805]: cluster 2024-02-18T01:12:44.855350+0000 osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:12:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:47 smithi155 bash[20805]: cluster 2024-02-18T01:12:44.855447+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:12:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:47 smithi155 bash[20805]: cluster 2024-02-18T01:12:46.621188+0000 mon.a (mon.0) 284 : cluster [INF] osd.1 v1:172.21.15.105:6805/1532722979 boot 2024-02-18T01:12:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:47 smithi155 bash[20805]: cluster 2024-02-18T01:12:46.621255+0000 mon.a (mon.0) 285 : cluster [DBG] osdmap e13: 2 total, 2 up, 2 in 2024-02-18T01:12:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:47 smithi155 bash[20805]: audit 2024-02-18T01:12:46.622690+0000 mon.a (mon.0) 286 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:12:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:48 smithi105 bash[27669]: cluster 2024-02-18T01:12:47.482490+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:48 smithi105 bash[27669]: cluster 2024-02-18T01:12:47.642003+0000 mon.a (mon.0) 287 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T01:12:48.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:48 smithi105 bash[19734]: cluster 2024-02-18T01:12:47.482490+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:48 smithi105 bash[19734]: cluster 2024-02-18T01:12:47.642003+0000 mon.a (mon.0) 287 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T01:12:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:48 smithi155 bash[20805]: cluster 2024-02-18T01:12:47.482490+0000 mgr.y (mgr.14146) 112 : cluster [DBG] pgmap v89: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-02-18T01:12:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:48 smithi155 bash[20805]: cluster 2024-02-18T01:12:47.642003+0000 mon.a (mon.0) 287 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2024-02-18T01:12:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T01:12:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T01:12:49.862 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-02-18T01:12:49.862 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-02-18T01:12:49.863 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0149633 s, 701 MB/s 2024-02-18T01:12:49.863 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-02-18T01:12:50.467 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_2 2024-02-18T01:12:50.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:50 smithi105 bash[27669]: cluster 2024-02-18T01:12:49.482887+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:50.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:50 smithi105 bash[19734]: cluster 2024-02-18T01:12:49.482887+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:50 smithi155 bash[20805]: cluster 2024-02-18T01:12:49.482887+0000 mgr.y (mgr.14146) 113 : cluster [DBG] pgmap v91: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:52 smithi105 bash[19734]: cluster 2024-02-18T01:12:51.483257+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:52.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:52 smithi105 bash[27669]: cluster 2024-02-18T01:12:51.483257+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:52 smithi155 bash[20805]: cluster 2024-02-18T01:12:51.483257+0000 mgr.y (mgr.14146) 114 : cluster [DBG] pgmap v92: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:54.506 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- 172.21.15.105:0/4269491832 >> v1:172.21.15.105:6789/0 conn(0x7ff2140fabf0 legacy=0x7ff2140fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:54.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- 172.21.15.105:0/4269491832 shutdown_connections 2024-02-18T01:12:54.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- 172.21.15.105:0/4269491832 >> 172.21.15.105:0/4269491832 conn(0x7ff2140f1e60 msgr2=0x7ff2140f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:12:54.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- 172.21.15.105:0/4269491832 shutdown_connections 2024-02-18T01:12:54.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- 172.21.15.105:0/4269491832 wait complete. 2024-02-18T01:12:54.507 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 Processor -- start 2024-02-18T01:12:54.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- start start 2024-02-18T01:12:54.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff2140614c0 con 0x7ff2140fe820 2024-02-18T01:12:54.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff214113a90 con 0x7ff2140fabf0 2024-02-18T01:12:54.508 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21bfba700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff214114c90 con 0x7ff2140f6720 2024-02-18T01:12:54.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff219d56700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7ff2140f6720 0x7ff21410a4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:39756/0 (socket says 172.21.15.105:39756) 2024-02-18T01:12:54.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff21a557700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ff2140fe820 0x7ff214111300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54144/0 (socket says 172.21.15.105:54144) 2024-02-18T01:12:54.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff219d56700 1 -- 172.21.15.105:0/2277635575 learned_addr learned my addr 172.21.15.105:0/2277635575 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:12:54.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4080358962 0 0) 0x7ff214103800 con 0x7ff2140fe820 2024-02-18T01:12:54.509 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1fc0033f0 con 0x7ff2140fe820 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086416078 0 0) 0x7ff214114c90 con 0x7ff2140f6720 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff214103800 con 0x7ff2140f6720 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4217086176 0 0) 0x7ff214113a90 con 0x7ff2140fabf0 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff214114c90 con 0x7ff2140fabf0 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1801156516 0 0) 0x7ff1fc0033f0 con 0x7ff2140fe820 2024-02-18T01:12:54.510 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff214113a90 con 0x7ff2140fe820 2024-02-18T01:12:54.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff2040016a0 con 0x7ff2140fe820 2024-02-18T01:12:54.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2950766486 0 0) 0x7ff214103800 con 0x7ff2140f6720 2024-02-18T01:12:54.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff1fc0033f0 con 0x7ff2140f6720 2024-02-18T01:12:54.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.498+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff20c001e20 con 0x7ff2140f6720 2024-02-18T01:12:54.511 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1243718404 0 0) 0x7ff214113a90 con 0x7ff2140fe820 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 >> v1:172.21.15.105:6790/0 conn(0x7ff2140f6720 legacy=0x7ff21410a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 >> v1:172.21.15.155:6789/0 conn(0x7ff2140fabf0 legacy=0x7ff21410dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff214115e90 con 0x7ff2140fe820 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff214114e80 con 0x7ff2140fe820 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2141152f0 con 0x7ff2140fe820 2024-02-18T01:12:54.512 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7ff2040021a0 con 0x7ff2140fe820 2024-02-18T01:12:54.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff204005850 con 0x7ff2140fe820 2024-02-18T01:12:54.513 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff200004f20 con 0x7ff2140fe820 2024-02-18T01:12:54.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7ff204001d00 con 0x7ff2140fe820 2024-02-18T01:12:54.514 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.502+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(14..14 src has 1..14) v4 ==== 1776+0+0 (unknown 2864129427 0 0) 0x7ff204076f70 con 0x7ff2140fe820 2024-02-18T01:12:54.516 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.510+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff204048130 con 0x7ff2140fe820 2024-02-18T01:12:54.707 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:12:54.702+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff200000b90 con 0x7ff1fc061fc0 2024-02-18T01:12:54.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:54 smithi105 bash[27669]: cluster 2024-02-18T01:12:53.483799+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:54.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:54 smithi105 bash[19734]: cluster 2024-02-18T01:12:53.483799+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:54 smithi155 bash[20805]: cluster 2024-02-18T01:12:53.483799+0000 mgr.y (mgr.14146) 115 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[19734]: audit 2024-02-18T01:12:54.709403+0000 mgr.y (mgr.14146) 116 : audit [DBG] from='client.14265 v1:172.21.15.105:0/2277635575' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[19734]: audit 2024-02-18T01:12:54.713276+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[19734]: audit 2024-02-18T01:12:54.716209+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[19734]: audit 2024-02-18T01:12:54.716887+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[27669]: audit 2024-02-18T01:12:54.709403+0000 mgr.y (mgr.14146) 116 : audit [DBG] from='client.14265 v1:172.21.15.105:0/2277635575' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[27669]: audit 2024-02-18T01:12:54.713276+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[27669]: audit 2024-02-18T01:12:54.716209+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:55 smithi105 bash[27669]: audit 2024-02-18T01:12:54.716887+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:55 smithi155 bash[20805]: audit 2024-02-18T01:12:54.709403+0000 mgr.y (mgr.14146) 116 : audit [DBG] from='client.14265 v1:172.21.15.105:0/2277635575' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:12:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:55 smithi155 bash[20805]: audit 2024-02-18T01:12:54.713276+0000 mon.a (mon.0) 288 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:12:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:55 smithi155 bash[20805]: audit 2024-02-18T01:12:54.716209+0000 mon.a (mon.0) 289 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:12:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:55 smithi155 bash[20805]: audit 2024-02-18T01:12:54.716887+0000 mon.a (mon.0) 290 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: cluster 2024-02-18T01:12:55.484265+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: cephadm 2024-02-18T01:12:55.865098+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.876633+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.878191+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.879426+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: cephadm 2024-02-18T01:12:55.880306+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 8147M 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.886434+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.887840+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.888921+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[19734]: audit 2024-02-18T01:12:55.896065+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: cluster 2024-02-18T01:12:55.484265+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: cephadm 2024-02-18T01:12:55.865098+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.876633+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.878191+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.879426+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: cephadm 2024-02-18T01:12:55.880306+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 8147M 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.886434+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.887840+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.888921+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:57.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:56 smithi105 bash[27669]: audit 2024-02-18T01:12:55.896065+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: cluster 2024-02-18T01:12:55.484265+0000 mgr.y (mgr.14146) 117 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: cephadm 2024-02-18T01:12:55.865098+0000 mgr.y (mgr.14146) 118 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:12:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.876633+0000 mon.a (mon.0) 291 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.878191+0000 mon.a (mon.0) 292 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.879426+0000 mon.a (mon.0) 293 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: cephadm 2024-02-18T01:12:55.880306+0000 mgr.y (mgr.14146) 119 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 8147M 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.886434+0000 mon.a (mon.0) 294 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.887840+0000 mon.a (mon.0) 295 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.888921+0000 mon.a (mon.0) 296 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:12:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:56 smithi155 bash[20805]: audit 2024-02-18T01:12:55.896065+0000 mon.a (mon.0) 297 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:12:59.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:12:58 smithi105 bash[27669]: cluster 2024-02-18T01:12:57.484805+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:59.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:12:58 smithi105 bash[19734]: cluster 2024-02-18T01:12:57.484805+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:12:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:12:58 smithi155 bash[20805]: cluster 2024-02-18T01:12:57.484805+0000 mgr.y (mgr.14146) 120 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:01.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:00 smithi105 bash[19734]: cluster 2024-02-18T01:12:59.485336+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:01.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:00 smithi105 bash[27669]: cluster 2024-02-18T01:12:59.485336+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:01.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:00 smithi155 bash[20805]: cluster 2024-02-18T01:12:59.485336+0000 mgr.y (mgr.14146) 121 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:03.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[19734]: cluster 2024-02-18T01:13:01.485802+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:03.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[19734]: audit 2024-02-18T01:13:02.773358+0000 mon.a (mon.0) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]: dispatch 2024-02-18T01:13:03.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[19734]: audit 2024-02-18T01:13:02.780858+0000 mon.a (mon.0) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]': finished 2024-02-18T01:13:03.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[19734]: cluster 2024-02-18T01:13:02.780901+0000 mon.a (mon.0) 300 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[19734]: audit 2024-02-18T01:13:02.781135+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[27669]: cluster 2024-02-18T01:13:01.485802+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[27669]: audit 2024-02-18T01:13:02.773358+0000 mon.a (mon.0) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]: dispatch 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[27669]: audit 2024-02-18T01:13:02.780858+0000 mon.a (mon.0) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]': finished 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[27669]: cluster 2024-02-18T01:13:02.780901+0000 mon.a (mon.0) 300 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T01:13:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:02 smithi105 bash[27669]: audit 2024-02-18T01:13:02.781135+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:02 smithi155 bash[20805]: cluster 2024-02-18T01:13:01.485802+0000 mgr.y (mgr.14146) 122 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:02 smithi155 bash[20805]: audit 2024-02-18T01:13:02.773358+0000 mon.a (mon.0) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]: dispatch 2024-02-18T01:13:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:02 smithi155 bash[20805]: audit 2024-02-18T01:13:02.780858+0000 mon.a (mon.0) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2609908915' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c08740fc-b607-4226-be20-1c56854e971c"}]': finished 2024-02-18T01:13:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:02 smithi155 bash[20805]: cluster 2024-02-18T01:13:02.780901+0000 mon.a (mon.0) 300 : cluster [DBG] osdmap e15: 3 total, 2 up, 3 in 2024-02-18T01:13:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:02 smithi155 bash[20805]: audit 2024-02-18T01:13:02.781135+0000 mon.a (mon.0) 301 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:04.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:03 smithi105 bash[19734]: audit 2024-02-18T01:13:03.386517+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? v1:172.21.15.105:0/3698792999' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:04.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:03 smithi105 bash[27669]: audit 2024-02-18T01:13:03.386517+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? v1:172.21.15.105:0/3698792999' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:04.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:03 smithi155 bash[20805]: audit 2024-02-18T01:13:03.386517+0000 mon.c (mon.2) 11 : audit [DBG] from='client.? v1:172.21.15.105:0/3698792999' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:05.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:04 smithi105 bash[19734]: cluster 2024-02-18T01:13:03.486143+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:05.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:04 smithi105 bash[27669]: cluster 2024-02-18T01:13:03.486143+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:04 smithi155 bash[20805]: cluster 2024-02-18T01:13:03.486143+0000 mgr.y (mgr.14146) 123 : cluster [DBG] pgmap v99: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:07.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:06 smithi105 bash[19734]: cluster 2024-02-18T01:13:05.486599+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:06 smithi105 bash[27669]: cluster 2024-02-18T01:13:05.486599+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:07.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:06 smithi155 bash[20805]: cluster 2024-02-18T01:13:05.486599+0000 mgr.y (mgr.14146) 124 : cluster [DBG] pgmap v100: 0 pgs: ; 0 B data, 580 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:09.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:08 smithi105 bash[19734]: cluster 2024-02-18T01:13:07.487132+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:09.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:08 smithi105 bash[27669]: cluster 2024-02-18T01:13:07.487132+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:08 smithi155 bash[20805]: cluster 2024-02-18T01:13:07.487132+0000 mgr.y (mgr.14146) 125 : cluster [DBG] pgmap v101: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:10 smithi105 bash[19734]: cluster 2024-02-18T01:13:09.487646+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:11.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:10 smithi105 bash[27669]: cluster 2024-02-18T01:13:09.487646+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:10 smithi155 bash[20805]: cluster 2024-02-18T01:13:09.487646+0000 mgr.y (mgr.14146) 126 : cluster [DBG] pgmap v102: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:13.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:12 smithi105 bash[19734]: cluster 2024-02-18T01:13:11.488239+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:12 smithi105 bash[27669]: cluster 2024-02-18T01:13:11.488239+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:12 smithi155 bash[20805]: cluster 2024-02-18T01:13:11.488239+0000 mgr.y (mgr.14146) 127 : cluster [DBG] pgmap v103: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[19734]: cluster 2024-02-18T01:13:13.488725+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[19734]: audit 2024-02-18T01:13:14.851046+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T01:13:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[19734]: audit 2024-02-18T01:13:14.852524+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:15.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[27669]: cluster 2024-02-18T01:13:13.488725+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[27669]: audit 2024-02-18T01:13:14.851046+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T01:13:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:14 smithi105 bash[27669]: audit 2024-02-18T01:13:14.852524+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:14 smithi155 bash[20805]: cluster 2024-02-18T01:13:13.488725+0000 mgr.y (mgr.14146) 128 : cluster [DBG] pgmap v104: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:15.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:14 smithi155 bash[20805]: audit 2024-02-18T01:13:14.851046+0000 mon.a (mon.0) 302 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-02-18T01:13:15.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:14 smithi155 bash[20805]: audit 2024-02-18T01:13:14.852524+0000 mon.a (mon.0) 303 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:16.206 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:15 smithi105 bash[19734]: cephadm 2024-02-18T01:13:14.853934+0000 mgr.y (mgr.14146) 129 : cephadm [INF] Deploying daemon osd.2 on smithi105 2024-02-18T01:13:16.207 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:15 smithi105 bash[27669]: cephadm 2024-02-18T01:13:14.853934+0000 mgr.y (mgr.14146) 129 : cephadm [INF] Deploying daemon osd.2 on smithi105 2024-02-18T01:13:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:15 smithi155 bash[20805]: cephadm 2024-02-18T01:13:14.853934+0000 mgr.y (mgr.14146) 129 : cephadm [INF] Deploying daemon osd.2 on smithi105 2024-02-18T01:13:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:16 smithi105 bash[19734]: cluster 2024-02-18T01:13:15.489236+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:16 smithi105 bash[27669]: cluster 2024-02-18T01:13:15.489236+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:16 smithi155 bash[20805]: cluster 2024-02-18T01:13:15.489236+0000 mgr.y (mgr.14146) 130 : cluster [DBG] pgmap v105: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[19734]: cluster 2024-02-18T01:13:17.489551+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[19734]: audit 2024-02-18T01:13:18.829739+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[19734]: audit 2024-02-18T01:13:18.873047+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[27669]: cluster 2024-02-18T01:13:17.489551+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[27669]: audit 2024-02-18T01:13:18.829739+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:18 smithi105 bash[27669]: audit 2024-02-18T01:13:18.873047+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:18 smithi155 bash[20805]: cluster 2024-02-18T01:13:17.489551+0000 mgr.y (mgr.14146) 131 : cluster [DBG] pgmap v106: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:18 smithi155 bash[20805]: audit 2024-02-18T01:13:18.829739+0000 mon.a (mon.0) 304 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:18 smithi155 bash[20805]: audit 2024-02-18T01:13:18.873047+0000 mon.a (mon.0) 305 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:20 smithi105 bash[19734]: cluster 2024-02-18T01:13:19.489984+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:21.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:20 smithi105 bash[27669]: cluster 2024-02-18T01:13:19.489984+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:20 smithi155 bash[20805]: cluster 2024-02-18T01:13:19.489984+0000 mgr.y (mgr.14146) 132 : cluster [DBG] pgmap v107: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:22 smithi105 bash[19734]: cluster 2024-02-18T01:13:21.490525+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:22 smithi105 bash[27669]: cluster 2024-02-18T01:13:21.490525+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:22 smithi155 bash[20805]: cluster 2024-02-18T01:13:21.490525+0000 mgr.y (mgr.14146) 133 : cluster [DBG] pgmap v108: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:24 smithi105 bash[27669]: cluster 2024-02-18T01:13:23.491040+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:25.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:24 smithi105 bash[19734]: cluster 2024-02-18T01:13:23.491040+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:24 smithi155 bash[20805]: cluster 2024-02-18T01:13:23.491040+0000 mgr.y (mgr.14146) 134 : cluster [DBG] pgmap v109: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:26.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.280421+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: cluster 2024-02-18T01:13:25.491343+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.558843+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.559953+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.566544+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.649605+0000 mon.c (mon.2) 12 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[19734]: audit 2024-02-18T01:13:25.649948+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.280421+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: cluster 2024-02-18T01:13:25.491343+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:26.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.558843+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:26.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.559953+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:26.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.566544+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.649605+0000 mon.c (mon.2) 12 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:26.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:26 smithi105 bash[27669]: audit 2024-02-18T01:13:25.649948+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.280421+0000 mon.a (mon.0) 306 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: cluster 2024-02-18T01:13:25.491343+0000 mgr.y (mgr.14146) 135 : cluster [DBG] pgmap v110: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.558843+0000 mon.a (mon.0) 307 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.559953+0000 mon.a (mon.0) 308 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.566544+0000 mon.a (mon.0) 309 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.649605+0000 mon.c (mon.2) 12 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:26 smithi155 bash[20805]: audit 2024-02-18T01:13:25.649948+0000 mon.a (mon.0) 310 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-02-18T01:13:27.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[27669]: audit 2024-02-18T01:13:26.571824+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T01:13:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[27669]: cluster 2024-02-18T01:13:26.572090+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T01:13:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[27669]: audit 2024-02-18T01:13:26.572331+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[27669]: audit 2024-02-18T01:13:26.573094+0000 mon.c (mon.2) 13 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[27669]: audit 2024-02-18T01:13:26.573732+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:27.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[19734]: audit 2024-02-18T01:13:26.571824+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T01:13:27.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[19734]: cluster 2024-02-18T01:13:26.572090+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T01:13:27.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[19734]: audit 2024-02-18T01:13:26.572331+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:27.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[19734]: audit 2024-02-18T01:13:26.573094+0000 mon.c (mon.2) 13 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:27.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:27 smithi105 bash[19734]: audit 2024-02-18T01:13:26.573732+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:27 smithi155 bash[20805]: audit 2024-02-18T01:13:26.571824+0000 mon.a (mon.0) 311 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-02-18T01:13:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:27 smithi155 bash[20805]: cluster 2024-02-18T01:13:26.572090+0000 mon.a (mon.0) 312 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2024-02-18T01:13:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:27 smithi155 bash[20805]: audit 2024-02-18T01:13:26.572331+0000 mon.a (mon.0) 313 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:27 smithi155 bash[20805]: audit 2024-02-18T01:13:26.573094+0000 mon.c (mon.2) 13 : audit [INF] from='osd.2 v1:172.21.15.105:6809/3935238244' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:27 smithi155 bash[20805]: audit 2024-02-18T01:13:26.573732+0000 mon.a (mon.0) 314 : audit [INF] from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:13:28.532 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.525+0000 7ff20affd700 1 -- 172.21.15.105:0/2277635575 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2805228497) 0x7ff200000b90 con 0x7ff1fc061fc0 2024-02-18T01:13:28.532 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 2 on host 'smithi105' 2024-02-18T01:13:28.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 >> v1:172.21.15.105:6800/3087430119 conn(0x7ff1fc061fc0 legacy=0x7ff1fc064480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:13:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 >> v1:172.21.15.105:6789/0 conn(0x7ff2140fe820 legacy=0x7ff214111300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:13:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 shutdown_connections 2024-02-18T01:13:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 >> 172.21.15.105:0/2277635575 conn(0x7ff2140f1e60 msgr2=0x7ff2140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:13:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 shutdown_connections 2024-02-18T01:13:28.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:28.529+0000 7ff21bfba700 1 -- 172.21.15.105:0/2277635575 wait complete. 2024-02-18T01:13:29.203 DEBUG:teuthology.orchestra.run.smithi105:osd.2> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.2.service 2024-02-18T01:13:29.206 INFO:tasks.cephadm:Deploying osd.3 on smithi105 with /dev/vg_nvme/lv_1... 2024-02-18T01:13:29.206 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T01:13:29.378 INFO:journalctl@ceph.osd.2.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: cluster 2024-02-18T01:13:27.491804+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: audit 2024-02-18T01:13:27.579020+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: cluster 2024-02-18T01:13:27.579174+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: audit 2024-02-18T01:13:27.579582+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: audit 2024-02-18T01:13:27.584419+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: audit 2024-02-18T01:13:28.530431+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[19734]: audit 2024-02-18T01:13:28.540438+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: cluster 2024-02-18T01:13:27.491804+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: audit 2024-02-18T01:13:27.579020+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: cluster 2024-02-18T01:13:27.579174+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: audit 2024-02-18T01:13:27.579582+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: audit 2024-02-18T01:13:27.584419+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: audit 2024-02-18T01:13:28.530431+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:29 smithi105 bash[27669]: audit 2024-02-18T01:13:28.540438+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: cluster 2024-02-18T01:13:27.491804+0000 mgr.y (mgr.14146) 136 : cluster [DBG] pgmap v112: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: audit 2024-02-18T01:13:27.579020+0000 mon.a (mon.0) 315 : audit [INF] from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:13:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: cluster 2024-02-18T01:13:27.579174+0000 mon.a (mon.0) 316 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2024-02-18T01:13:29.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: audit 2024-02-18T01:13:27.579582+0000 mon.a (mon.0) 317 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: audit 2024-02-18T01:13:27.584419+0000 mon.a (mon.0) 318 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:29.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: audit 2024-02-18T01:13:28.530431+0000 mon.a (mon.0) 319 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:29.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:29 smithi155 bash[20805]: audit 2024-02-18T01:13:28.540438+0000 mon.a (mon.0) 320 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: cluster 2024-02-18T01:13:26.607165+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: cluster 2024-02-18T01:13:26.607266+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: cluster 2024-02-18T01:13:28.585207+0000 mon.a (mon.0) 321 : cluster [INF] osd.2 v1:172.21.15.105:6809/3935238244 boot 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: cluster 2024-02-18T01:13:28.585284+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: audit 2024-02-18T01:13:28.586119+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: audit 2024-02-18T01:13:28.586745+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: audit 2024-02-18T01:13:29.583946+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: cluster 2024-02-18T01:13:29.584024+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T01:13:30.315 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[27669]: audit 2024-02-18T01:13:29.585920+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: cluster 2024-02-18T01:13:26.607165+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: cluster 2024-02-18T01:13:26.607266+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: cluster 2024-02-18T01:13:28.585207+0000 mon.a (mon.0) 321 : cluster [INF] osd.2 v1:172.21.15.105:6809/3935238244 boot 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: cluster 2024-02-18T01:13:28.585284+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: audit 2024-02-18T01:13:28.586119+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: audit 2024-02-18T01:13:28.586745+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: audit 2024-02-18T01:13:29.583946+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: cluster 2024-02-18T01:13:29.584024+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T01:13:30.316 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:30 smithi105 bash[19734]: audit 2024-02-18T01:13:29.585920+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:13:30.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: cluster 2024-02-18T01:13:26.607165+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: cluster 2024-02-18T01:13:26.607266+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: cluster 2024-02-18T01:13:28.585207+0000 mon.a (mon.0) 321 : cluster [INF] osd.2 v1:172.21.15.105:6809/3935238244 boot 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: cluster 2024-02-18T01:13:28.585284+0000 mon.a (mon.0) 322 : cluster [DBG] osdmap e18: 3 total, 3 up, 3 in 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: audit 2024-02-18T01:13:28.586119+0000 mon.a (mon.0) 323 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: audit 2024-02-18T01:13:28.586745+0000 mon.a (mon.0) 324 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: audit 2024-02-18T01:13:29.583946+0000 mon.a (mon.0) 325 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: cluster 2024-02-18T01:13:29.584024+0000 mon.a (mon.0) 326 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2024-02-18T01:13:30.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:30 smithi155 bash[20805]: audit 2024-02-18T01:13:29.585920+0000 mon.a (mon.0) 327 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:13:31.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[27669]: cluster 2024-02-18T01:13:29.492220+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:31.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[27669]: audit 2024-02-18T01:13:30.589360+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:13:31.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[27669]: cluster 2024-02-18T01:13:30.589634+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T01:13:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[19734]: cluster 2024-02-18T01:13:29.492220+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[19734]: audit 2024-02-18T01:13:30.589360+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:13:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:31 smithi105 bash[19734]: cluster 2024-02-18T01:13:30.589634+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T01:13:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:31 smithi155 bash[20805]: cluster 2024-02-18T01:13:29.492220+0000 mgr.y (mgr.14146) 137 : cluster [DBG] pgmap v115: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-02-18T01:13:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:31 smithi155 bash[20805]: audit 2024-02-18T01:13:30.589360+0000 mon.a (mon.0) 328 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:13:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:31 smithi155 bash[20805]: cluster 2024-02-18T01:13:30.589634+0000 mon.a (mon.0) 329 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10+0 records in 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:10+0 records out 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0150801 s, 695 MB/s 2024-02-18T01:13:32.675 INFO:teuthology.orchestra.run.smithi105.stderr:--> Zapping successful for: 2024-02-18T01:13:32.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:32 smithi105 bash[27669]: cluster 2024-02-18T01:13:31.492713+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v118: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:32.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:32 smithi105 bash[27669]: cluster 2024-02-18T01:13:31.588151+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T01:13:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:32 smithi105 bash[19734]: cluster 2024-02-18T01:13:31.492713+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v118: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:32.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:32 smithi105 bash[19734]: cluster 2024-02-18T01:13:31.588151+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T01:13:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:32 smithi155 bash[20805]: cluster 2024-02-18T01:13:31.492713+0000 mgr.y (mgr.14146) 138 : cluster [DBG] pgmap v118: 1 pgs: 1 unknown; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:32 smithi155 bash[20805]: cluster 2024-02-18T01:13:31.588151+0000 mon.a (mon.0) 330 : cluster [DBG] osdmap e21: 3 total, 3 up, 3 in 2024-02-18T01:13:33.385 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi105:vg_nvme/lv_1 2024-02-18T01:13:34.918 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:34 smithi105 bash[19734]: cluster 2024-02-18T01:13:33.493249+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:34.918 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:34 smithi105 bash[27669]: cluster 2024-02-18T01:13:33.493249+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:34.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:34 smithi155 bash[20805]: cluster 2024-02-18T01:13:33.493249+0000 mgr.y (mgr.14146) 139 : cluster [DBG] pgmap v120: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:36.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:36 smithi105 bash[19734]: cluster 2024-02-18T01:13:35.493814+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:36.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:36 smithi105 bash[27669]: cluster 2024-02-18T01:13:35.493814+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:36 smithi155 bash[20805]: cluster 2024-02-18T01:13:35.493814+0000 mgr.y (mgr.14146) 140 : cluster [DBG] pgmap v121: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- 172.21.15.105:0/395305036 >> v1:172.21.15.105:6789/0 conn(0x7fbedc101b20 legacy=0x7fbedc103fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:13:37.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- 172.21.15.105:0/395305036 shutdown_connections 2024-02-18T01:13:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- 172.21.15.105:0/395305036 >> 172.21.15.105:0/395305036 conn(0x7fbedc0f1e60 msgr2=0x7fbedc0f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:13:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- 172.21.15.105:0/395305036 shutdown_connections 2024-02-18T01:13:37.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- 172.21.15.105:0/395305036 wait complete. 2024-02-18T01:13:37.548 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 Processor -- start 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- start start 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbedc10a330 con 0x7fbedc101b20 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbedc115b90 con 0x7fbedc061240 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee344b700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbedc116d70 con 0x7fbedc0fdf20 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee09e6700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fbedc0fdf20 0x7fbedc10fd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:34606/0 (socket says 172.21.15.105:34606) 2024-02-18T01:13:37.549 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbee09e6700 1 -- 172.21.15.105:0/1247448706 learned_addr learned my addr 172.21.15.105:0/1247448706 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:13:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1022989601 0 0) 0x7fbedc10a330 con 0x7fbedc101b20 2024-02-18T01:13:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbec8003110 con 0x7fbedc101b20 2024-02-18T01:13:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464197138 0 0) 0x7fbedc116d70 con 0x7fbedc0fdf20 2024-02-18T01:13:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbedc10a330 con 0x7fbedc0fdf20 2024-02-18T01:13:37.550 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3534964762 0 0) 0x7fbedc115b90 con 0x7fbedc061240 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbedc116d70 con 0x7fbedc061240 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2817599916 0 0) 0x7fbedc10a330 con 0x7fbedc0fdf20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbedc115b90 con 0x7fbedc0fdf20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.537+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbed8001a70 con 0x7fbedc0fdf20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2133960511 0 0) 0x7fbec8003110 con 0x7fbedc101b20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbedc10a330 con 0x7fbedc101b20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbed4002380 con 0x7fbedc101b20 2024-02-18T01:13:37.551 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4275409029 0 0) 0x7fbedc116d70 con 0x7fbedc061240 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbec8003110 con 0x7fbedc061240 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbecc001840 con 0x7fbedc061240 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1338185506 0 0) 0x7fbedc10a330 con 0x7fbedc101b20 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 >> v1:172.21.15.105:6790/0 conn(0x7fbedc0fdf20 legacy=0x7fbedc10fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 >> v1:172.21.15.155:6789/0 conn(0x7fbedc061240 legacy=0x7fbedc109760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbedc117f50 con 0x7fbedc101b20 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbedc116f60 con 0x7fbedc101b20 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fbed4002510 con 0x7fbedc101b20 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbed4003b90 con 0x7fbedc101b20 2024-02-18T01:13:37.552 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbedc117450 con 0x7fbedc101b20 2024-02-18T01:13:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbedc047540 con 0x7fbedc101b20 2024-02-18T01:13:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7fbed4002f20 con 0x7fbedc101b20 2024-02-18T01:13:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.541+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(21..21 src has 1..21) v4 ==== 2484+0+0 (unknown 3222818430 0 0) 0x7fbed4075520 con 0x7fbedc101b20 2024-02-18T01:13:37.553 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.545+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fbed4046480 con 0x7fbedc101b20 2024-02-18T01:13:37.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:13:37.741+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbedc0fd640 con 0x7fbec8062220 2024-02-18T01:13:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[19734]: cluster 2024-02-18T01:13:37.494233+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[19734]: audit 2024-02-18T01:13:37.748158+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.14292 v1:172.21.15.105:0/1247448706' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:13:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[19734]: audit 2024-02-18T01:13:37.754918+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:13:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[19734]: audit 2024-02-18T01:13:37.758441+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[19734]: audit 2024-02-18T01:13:37.759187+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[27669]: cluster 2024-02-18T01:13:37.494233+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[27669]: audit 2024-02-18T01:13:37.748158+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.14292 v1:172.21.15.105:0/1247448706' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[27669]: audit 2024-02-18T01:13:37.754918+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[27669]: audit 2024-02-18T01:13:37.758441+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:13:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:38 smithi105 bash[27669]: audit 2024-02-18T01:13:37.759187+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:38 smithi155 bash[20805]: cluster 2024-02-18T01:13:37.494233+0000 mgr.y (mgr.14146) 141 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:38 smithi155 bash[20805]: audit 2024-02-18T01:13:37.748158+0000 mgr.y (mgr.14146) 142 : audit [DBG] from='client.14292 v1:172.21.15.105:0/1247448706' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi105:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:13:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:38 smithi155 bash[20805]: audit 2024-02-18T01:13:37.754918+0000 mon.a (mon.0) 331 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:13:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:38 smithi155 bash[20805]: audit 2024-02-18T01:13:37.758441+0000 mon.a (mon.0) 332 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:13:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:38 smithi155 bash[20805]: audit 2024-02-18T01:13:37.759187+0000 mon.a (mon.0) 333 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:40.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: cephadm 2024-02-18T01:13:39.145700+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:13:40.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.155784+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.157847+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.159392+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.160699+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: cephadm 2024-02-18T01:13:39.161765+0000 mgr.y (mgr.14146) 144 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 5431M 2024-02-18T01:13:40.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.169348+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.170861+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:40.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.171861+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:40.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: audit 2024-02-18T01:13:39.179519+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[19734]: cluster 2024-02-18T01:13:39.494745+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:40.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: cephadm 2024-02-18T01:13:39.145700+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:13:40.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.155784+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.157847+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.159392+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.160699+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: cephadm 2024-02-18T01:13:39.161765+0000 mgr.y (mgr.14146) 144 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 5431M 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.169348+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.170861+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.171861+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: audit 2024-02-18T01:13:39.179519+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:40 smithi105 bash[27669]: cluster 2024-02-18T01:13:39.494745+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:40.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: cephadm 2024-02-18T01:13:39.145700+0000 mgr.y (mgr.14146) 143 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:13:40.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.155784+0000 mon.a (mon.0) 334 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.157847+0000 mon.a (mon.0) 335 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.159392+0000 mon.a (mon.0) 336 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.160699+0000 mon.a (mon.0) 337 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: cephadm 2024-02-18T01:13:39.161765+0000 mgr.y (mgr.14146) 144 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 5431M 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.169348+0000 mon.a (mon.0) 338 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.170861+0000 mon.a (mon.0) 339 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:40.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.171861+0000 mon.a (mon.0) 340 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:13:40.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: audit 2024-02-18T01:13:39.179519+0000 mon.a (mon.0) 341 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:13:40.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:40 smithi155 bash[20805]: cluster 2024-02-18T01:13:39.494745+0000 mgr.y (mgr.14146) 145 : cluster [DBG] pgmap v123: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:42 smithi105 bash[27669]: cluster 2024-02-18T01:13:41.495284+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:42 smithi105 bash[19734]: cluster 2024-02-18T01:13:41.495284+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:42 smithi155 bash[20805]: cluster 2024-02-18T01:13:41.495284+0000 mgr.y (mgr.14146) 146 : cluster [DBG] pgmap v124: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:44 smithi105 bash[19734]: cluster 2024-02-18T01:13:43.495764+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:44 smithi105 bash[27669]: cluster 2024-02-18T01:13:43.495764+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:44 smithi155 bash[20805]: cluster 2024-02-18T01:13:43.495764+0000 mgr.y (mgr.14146) 147 : cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:46.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[19734]: cluster 2024-02-18T01:13:45.496106+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:46.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[19734]: audit 2024-02-18T01:13:45.959164+0000 mon.a (mon.0) 342 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]: dispatch 2024-02-18T01:13:46.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[19734]: audit 2024-02-18T01:13:45.970371+0000 mon.a (mon.0) 343 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]': finished 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[19734]: cluster 2024-02-18T01:13:45.970482+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[19734]: audit 2024-02-18T01:13:45.970634+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[27669]: cluster 2024-02-18T01:13:45.496106+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[27669]: audit 2024-02-18T01:13:45.959164+0000 mon.a (mon.0) 342 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]: dispatch 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[27669]: audit 2024-02-18T01:13:45.970371+0000 mon.a (mon.0) 343 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]': finished 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[27669]: cluster 2024-02-18T01:13:45.970482+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T01:13:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:46 smithi105 bash[27669]: audit 2024-02-18T01:13:45.970634+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:13:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:46 smithi155 bash[20805]: cluster 2024-02-18T01:13:45.496106+0000 mgr.y (mgr.14146) 148 : cluster [DBG] pgmap v126: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:46 smithi155 bash[20805]: audit 2024-02-18T01:13:45.959164+0000 mon.a (mon.0) 342 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]: dispatch 2024-02-18T01:13:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:46 smithi155 bash[20805]: audit 2024-02-18T01:13:45.970371+0000 mon.a (mon.0) 343 : audit [INF] from='client.? v1:172.21.15.105:0/4016557324' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "eaef36ae-a5ea-4a4a-8c5f-c88098a48611"}]': finished 2024-02-18T01:13:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:46 smithi155 bash[20805]: cluster 2024-02-18T01:13:45.970482+0000 mon.a (mon.0) 344 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2024-02-18T01:13:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:46 smithi155 bash[20805]: audit 2024-02-18T01:13:45.970634+0000 mon.a (mon.0) 345 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:13:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:47 smithi105 bash[19734]: audit 2024-02-18T01:13:46.592862+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? v1:172.21.15.105:0/3838671288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:47.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:47 smithi105 bash[27669]: audit 2024-02-18T01:13:46.592862+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? v1:172.21.15.105:0/3838671288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:47 smithi155 bash[20805]: audit 2024-02-18T01:13:46.592862+0000 mon.c (mon.2) 14 : audit [DBG] from='client.? v1:172.21.15.105:0/3838671288' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:13:48.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:48 smithi105 bash[19734]: cluster 2024-02-18T01:13:47.496682+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:48 smithi105 bash[27669]: cluster 2024-02-18T01:13:47.496682+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:48 smithi155 bash[20805]: cluster 2024-02-18T01:13:47.496682+0000 mgr.y (mgr.14146) 149 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:50.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:50 smithi105 bash[19734]: cluster 2024-02-18T01:13:49.497451+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:50.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:50 smithi105 bash[27669]: cluster 2024-02-18T01:13:49.497451+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:50 smithi155 bash[20805]: cluster 2024-02-18T01:13:49.497451+0000 mgr.y (mgr.14146) 150 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:52 smithi105 bash[19734]: cluster 2024-02-18T01:13:51.498020+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:52.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:52 smithi105 bash[27669]: cluster 2024-02-18T01:13:51.498020+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:52.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:52 smithi155 bash[20805]: cluster 2024-02-18T01:13:51.498020+0000 mgr.y (mgr.14146) 151 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:54.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:54 smithi105 bash[19734]: cluster 2024-02-18T01:13:53.498596+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:54.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:54 smithi105 bash[27669]: cluster 2024-02-18T01:13:53.498596+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:54 smithi155 bash[20805]: cluster 2024-02-18T01:13:53.498596+0000 mgr.y (mgr.14146) 152 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:56.938 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:56 smithi105 bash[19734]: cluster 2024-02-18T01:13:55.499237+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:56.939 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:56 smithi105 bash[27669]: cluster 2024-02-18T01:13:55.499237+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:56 smithi155 bash[20805]: cluster 2024-02-18T01:13:55.499237+0000 mgr.y (mgr.14146) 153 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:58.946 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[27669]: cluster 2024-02-18T01:13:57.499819+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:58.946 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[27669]: audit 2024-02-18T01:13:58.022468+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[27669]: audit 2024-02-18T01:13:58.023882+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[27669]: cephadm 2024-02-18T01:13:58.025050+0000 mgr.y (mgr.14146) 155 : cephadm [INF] Deploying daemon osd.3 on smithi105 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[19734]: cluster 2024-02-18T01:13:57.499819+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[19734]: audit 2024-02-18T01:13:58.022468+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[19734]: audit 2024-02-18T01:13:58.023882+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:58.947 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:13:58 smithi105 bash[19734]: cephadm 2024-02-18T01:13:58.025050+0000 mgr.y (mgr.14146) 155 : cephadm [INF] Deploying daemon osd.3 on smithi105 2024-02-18T01:13:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:58 smithi155 bash[20805]: cluster 2024-02-18T01:13:57.499819+0000 mgr.y (mgr.14146) 154 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:13:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:58 smithi155 bash[20805]: audit 2024-02-18T01:13:58.022468+0000 mon.a (mon.0) 346 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-02-18T01:13:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:58 smithi155 bash[20805]: audit 2024-02-18T01:13:58.023882+0000 mon.a (mon.0) 347 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:13:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:13:58 smithi155 bash[20805]: cephadm 2024-02-18T01:13:58.025050+0000 mgr.y (mgr.14146) 155 : cephadm [INF] Deploying daemon osd.3 on smithi105 2024-02-18T01:14:00.964 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:00 smithi105 bash[19734]: cluster 2024-02-18T01:13:59.500398+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:00.965 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:00 smithi105 bash[27669]: cluster 2024-02-18T01:13:59.500398+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:00.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:00 smithi155 bash[20805]: cluster 2024-02-18T01:13:59.500398+0000 mgr.y (mgr.14146) 156 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[27669]: cluster 2024-02-18T01:14:01.500883+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[27669]: audit 2024-02-18T01:14:01.826018+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[27669]: audit 2024-02-18T01:14:01.850980+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[19734]: cluster 2024-02-18T01:14:01.500883+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[19734]: audit 2024-02-18T01:14:01.826018+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:03.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:02 smithi105 bash[19734]: audit 2024-02-18T01:14:01.850980+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:02 smithi155 bash[20805]: cluster 2024-02-18T01:14:01.500883+0000 mgr.y (mgr.14146) 157 : cluster [DBG] pgmap v135: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:02 smithi155 bash[20805]: audit 2024-02-18T01:14:01.826018+0000 mon.a (mon.0) 348 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:02 smithi155 bash[20805]: audit 2024-02-18T01:14:01.850980+0000 mon.a (mon.0) 349 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:04.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:04 smithi105 bash[19734]: cluster 2024-02-18T01:14:03.501390+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:04.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:04 smithi105 bash[27669]: cluster 2024-02-18T01:14:03.501390+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:04 smithi155 bash[20805]: cluster 2024-02-18T01:14:03.501390+0000 mgr.y (mgr.14146) 158 : cluster [DBG] pgmap v136: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:07.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:06 smithi105 bash[27669]: cluster 2024-02-18T01:14:05.501861+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:07.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:06 smithi105 bash[19734]: cluster 2024-02-18T01:14:05.501861+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:07.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:06 smithi155 bash[20805]: cluster 2024-02-18T01:14:05.501861+0000 mgr.y (mgr.14146) 159 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:08.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:07.419430+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:07.420570+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:07.421609+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:07.427669+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: cluster 2024-02-18T01:14:07.502266+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:08.332018+0000 mon.c (mon.2) 15 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[19734]: audit 2024-02-18T01:14:08.332324+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:07.419430+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:07.420570+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:07.421609+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:07.427669+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: cluster 2024-02-18T01:14:07.502266+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:08.332018+0000 mon.c (mon.2) 15 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:08.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:08 smithi105 bash[27669]: audit 2024-02-18T01:14:08.332324+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:08.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:07.419430+0000 mon.a (mon.0) 350 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:07.420570+0000 mon.a (mon.0) 351 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:07.421609+0000 mon.a (mon.0) 352 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:07.427669+0000 mon.a (mon.0) 353 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: cluster 2024-02-18T01:14:07.502266+0000 mgr.y (mgr.14146) 160 : cluster [DBG] pgmap v138: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:08.332018+0000 mon.c (mon.2) 15 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:08 smithi155 bash[20805]: audit 2024-02-18T01:14:08.332324+0000 mon.a (mon.0) 354 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-02-18T01:14:09.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[27669]: audit 2024-02-18T01:14:08.431555+0000 mon.a (mon.0) 355 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T01:14:09.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[27669]: cluster 2024-02-18T01:14:08.431707+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T01:14:09.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[27669]: audit 2024-02-18T01:14:08.432108+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:09.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[27669]: audit 2024-02-18T01:14:08.432863+0000 mon.c (mon.2) 16 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:09.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[27669]: audit 2024-02-18T01:14:08.435757+0000 mon.a (mon.0) 358 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[19734]: audit 2024-02-18T01:14:08.431555+0000 mon.a (mon.0) 355 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T01:14:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[19734]: cluster 2024-02-18T01:14:08.431707+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T01:14:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[19734]: audit 2024-02-18T01:14:08.432108+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[19734]: audit 2024-02-18T01:14:08.432863+0000 mon.c (mon.2) 16 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:09 smithi105 bash[19734]: audit 2024-02-18T01:14:08.435757+0000 mon.a (mon.0) 358 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:09.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:09 smithi155 bash[20805]: audit 2024-02-18T01:14:08.431555+0000 mon.a (mon.0) 355 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-02-18T01:14:09.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:09 smithi155 bash[20805]: cluster 2024-02-18T01:14:08.431707+0000 mon.a (mon.0) 356 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2024-02-18T01:14:09.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:09 smithi155 bash[20805]: audit 2024-02-18T01:14:08.432108+0000 mon.a (mon.0) 357 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:09.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:09 smithi155 bash[20805]: audit 2024-02-18T01:14:08.432863+0000 mon.c (mon.2) 16 : audit [INF] from='osd.3 v1:172.21.15.105:6813/1099653553' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:09.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:09 smithi155 bash[20805]: audit 2024-02-18T01:14:08.435757+0000 mon.a (mon.0) 358 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]: dispatch 2024-02-18T01:14:10.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.664+0000 7fbed27fc700 1 -- 172.21.15.105:0/1247448706 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 4213280101) 0x7fbedc0fd640 con 0x7fbec8062220 2024-02-18T01:14:10.669 INFO:teuthology.orchestra.run.smithi105.stdout:Created osd(s) 3 on host 'smithi105' 2024-02-18T01:14:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 >> v1:172.21.15.105:6800/3087430119 conn(0x7fbec8062220 legacy=0x7fbec80646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:10.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 >> v1:172.21.15.105:6789/0 conn(0x7fbedc101b20 legacy=0x7fbedc113460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 shutdown_connections 2024-02-18T01:14:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 >> 172.21.15.105:0/1247448706 conn(0x7fbedc0f1e60 msgr2=0x7fbedc103fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:14:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 shutdown_connections 2024-02-18T01:14:10.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:14:10.668+0000 7fbee344b700 1 -- 172.21.15.105:0/1247448706 wait complete. 2024-02-18T01:14:10.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[19734]: audit 2024-02-18T01:14:09.445092+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:14:10.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[19734]: cluster 2024-02-18T01:14:09.445221+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T01:14:10.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[19734]: audit 2024-02-18T01:14:09.445398+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[19734]: audit 2024-02-18T01:14:09.449770+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[19734]: cluster 2024-02-18T01:14:09.502780+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[27669]: audit 2024-02-18T01:14:09.445092+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:14:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[27669]: cluster 2024-02-18T01:14:09.445221+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T01:14:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[27669]: audit 2024-02-18T01:14:09.445398+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[27669]: audit 2024-02-18T01:14:09.449770+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:10 smithi105 bash[27669]: cluster 2024-02-18T01:14:09.502780+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:10.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:10 smithi155 bash[20805]: audit 2024-02-18T01:14:09.445092+0000 mon.a (mon.0) 359 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi105", "root=default"]}]': finished 2024-02-18T01:14:10.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:10 smithi155 bash[20805]: cluster 2024-02-18T01:14:09.445221+0000 mon.a (mon.0) 360 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2024-02-18T01:14:10.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:10 smithi155 bash[20805]: audit 2024-02-18T01:14:09.445398+0000 mon.a (mon.0) 361 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:10 smithi155 bash[20805]: audit 2024-02-18T01:14:09.449770+0000 mon.a (mon.0) 362 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:10 smithi155 bash[20805]: cluster 2024-02-18T01:14:09.502780+0000 mgr.y (mgr.14146) 161 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 871 MiB used, 267 GiB / 268 GiB avail 2024-02-18T01:14:11.448 DEBUG:teuthology.orchestra.run.smithi105:osd.3> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.3.service 2024-02-18T01:14:11.450 INFO:tasks.cephadm:Deploying osd.4 on smithi155 with /dev/vg_nvme/lv_4... 2024-02-18T01:14:11.450 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-02-18T01:14:11.571 INFO:journalctl@ceph.osd.3.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:14:11.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: cluster 2024-02-18T01:14:09.302087+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:11.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: cluster 2024-02-18T01:14:09.302202+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: cluster 2024-02-18T01:14:10.449833+0000 mon.a (mon.0) 363 : cluster [INF] osd.3 v1:172.21.15.105:6813/1099653553 boot 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: cluster 2024-02-18T01:14:10.449913+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: audit 2024-02-18T01:14:10.451231+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: audit 2024-02-18T01:14:10.667243+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[19734]: audit 2024-02-18T01:14:10.702216+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: cluster 2024-02-18T01:14:09.302087+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: cluster 2024-02-18T01:14:09.302202+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: cluster 2024-02-18T01:14:10.449833+0000 mon.a (mon.0) 363 : cluster [INF] osd.3 v1:172.21.15.105:6813/1099653553 boot 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: cluster 2024-02-18T01:14:10.449913+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: audit 2024-02-18T01:14:10.451231+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: audit 2024-02-18T01:14:10.667243+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:11 smithi105 bash[27669]: audit 2024-02-18T01:14:10.702216+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:11.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: cluster 2024-02-18T01:14:09.302087+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:11.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: cluster 2024-02-18T01:14:09.302202+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: cluster 2024-02-18T01:14:10.449833+0000 mon.a (mon.0) 363 : cluster [INF] osd.3 v1:172.21.15.105:6813/1099653553 boot 2024-02-18T01:14:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: cluster 2024-02-18T01:14:10.449913+0000 mon.a (mon.0) 364 : cluster [DBG] osdmap e25: 4 total, 4 up, 4 in 2024-02-18T01:14:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: audit 2024-02-18T01:14:10.451231+0000 mon.a (mon.0) 365 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:14:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: audit 2024-02-18T01:14:10.667243+0000 mon.a (mon.0) 366 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:11 smithi155 bash[20805]: audit 2024-02-18T01:14:10.702216+0000 mon.a (mon.0) 367 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:12 smithi155 bash[20805]: cluster 2024-02-18T01:14:11.503357+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:12 smithi155 bash[20805]: cluster 2024-02-18T01:14:11.670089+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T01:14:12.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:12 smithi105 bash[19734]: cluster 2024-02-18T01:14:11.503357+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:12 smithi105 bash[19734]: cluster 2024-02-18T01:14:11.670089+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T01:14:12.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:12 smithi105 bash[27669]: cluster 2024-02-18T01:14:11.503357+0000 mgr.y (mgr.14146) 162 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:12.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:12 smithi105 bash[27669]: cluster 2024-02-18T01:14:11.670089+0000 mon.a (mon.0) 368 : cluster [DBG] osdmap e26: 4 total, 4 up, 4 in 2024-02-18T01:14:14.429 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-02-18T01:14:14.430 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-02-18T01:14:14.430 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-18T01:14:14.430 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-18T01:14:14.430 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134665 s, 779 MB/s 2024-02-18T01:14:14.430 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-18T01:14:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:14 smithi105 bash[19734]: cluster 2024-02-18T01:14:13.503946+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:14 smithi105 bash[27669]: cluster 2024-02-18T01:14:13.503946+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:14 smithi155 bash[20805]: cluster 2024-02-18T01:14:13.503946+0000 mgr.y (mgr.14146) 163 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:15.025 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_4 2024-02-18T01:14:16.726 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:16 smithi105 bash[19734]: cluster 2024-02-18T01:14:15.504550+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:16.726 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:16 smithi105 bash[27669]: cluster 2024-02-18T01:14:15.504550+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:16 smithi155 bash[20805]: cluster 2024-02-18T01:14:15.504550+0000 mgr.y (mgr.14146) 164 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:17.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- 172.21.15.155:0/496881348 >> v1:172.21.15.155:6789/0 conn(0x7f233c0fbbf0 legacy=0x7f233c0fe060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:17.739 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- 172.21.15.155:0/496881348 shutdown_connections 2024-02-18T01:14:17.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- 172.21.15.155:0/496881348 >> 172.21.15.155:0/496881348 conn(0x7f233c0f1eb0 msgr2=0x7f233c0f42d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:14:17.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- 172.21.15.155:0/496881348 shutdown_connections 2024-02-18T01:14:17.740 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- 172.21.15.155:0/496881348 wait complete. 2024-02-18T01:14:17.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 Processor -- start 2024-02-18T01:14:17.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- start start 2024-02-18T01:14:17.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f233c108570 con 0x7f233c0ff920 2024-02-18T01:14:17.744 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f233c113dd0 con 0x7f233c0fbbf0 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f23430d7700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f233c114fb0 con 0x7f233c061660 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f233bfff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f233c0fbbf0 0x7f233c10df70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:36892/0 (socket says 172.21.15.155:36892) 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f233bfff700 1 -- 172.21.15.155:0/580284063 learned_addr learned my addr 172.21.15.155:0/580284063 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3981907860 0 0) 0x7f233c113dd0 con 0x7f233c0fbbf0 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2328003030 con 0x7f233c0fbbf0 2024-02-18T01:14:17.745 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945235638 0 0) 0x7f233c114fb0 con 0x7f233c061660 2024-02-18T01:14:17.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233c113dd0 con 0x7f233c061660 2024-02-18T01:14:17.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1861631482 0 0) 0x7f233c108570 con 0x7f233c0ff920 2024-02-18T01:14:17.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f233c114fb0 con 0x7f233c0ff920 2024-02-18T01:14:17.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1924720092 0 0) 0x7f233c113dd0 con 0x7f233c061660 2024-02-18T01:14:17.746 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f233c108570 con 0x7f233c061660 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f232c0017b0 con 0x7f233c061660 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2254152305 0 0) 0x7f2328003030 con 0x7f233c0fbbf0 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f233c113dd0 con 0x7f233c0fbbf0 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f23240015d0 con 0x7f233c0fbbf0 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 573026357 0 0) 0x7f233c114fb0 con 0x7f233c0ff920 2024-02-18T01:14:17.747 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2328003030 con 0x7f233c0ff920 2024-02-18T01:14:17.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.735+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2334001b80 con 0x7f233c0ff920 2024-02-18T01:14:17.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2373070464 0 0) 0x7f233c108570 con 0x7f233c061660 2024-02-18T01:14:17.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 >> v1:172.21.15.155:6789/0 conn(0x7f233c0fbbf0 legacy=0x7f233c10df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:17.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 >> v1:172.21.15.105:6789/0 conn(0x7f233c0ff920 legacy=0x7f233c1116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:17.748 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233c116190 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233c1151a0 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f233c115690 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f232c001e70 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f232c003230 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c004f20 con 0x7f233c061660 2024-02-18T01:14:17.749 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f232c0019d0 con 0x7f233c061660 2024-02-18T01:14:17.750 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.739+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(26..26 src has 1..26) v4 ==== 2776+0+0 (unknown 935328113 0 0) 0x7f232c074e70 con 0x7f233c061660 2024-02-18T01:14:17.752 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.747+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f232c045bf0 con 0x7f233c061660 2024-02-18T01:14:17.939 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:17.935+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f231c000b90 con 0x7f2328062340 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[27669]: cluster 2024-02-18T01:14:17.505098+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[27669]: audit 2024-02-18T01:14:17.941399+0000 mgr.y (mgr.14146) 166 : audit [DBG] from='client.24233 v1:172.21.15.155:0/580284063' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[27669]: audit 2024-02-18T01:14:17.948011+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[27669]: audit 2024-02-18T01:14:17.952704+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[27669]: audit 2024-02-18T01:14:17.954094+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[19734]: cluster 2024-02-18T01:14:17.505098+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:18.953 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[19734]: audit 2024-02-18T01:14:17.941399+0000 mgr.y (mgr.14146) 166 : audit [DBG] from='client.24233 v1:172.21.15.155:0/580284063' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:14:18.954 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[19734]: audit 2024-02-18T01:14:17.948011+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:18.954 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[19734]: audit 2024-02-18T01:14:17.952704+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:18.954 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:18 smithi105 bash[19734]: audit 2024-02-18T01:14:17.954094+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:18 smithi155 bash[20805]: cluster 2024-02-18T01:14:17.505098+0000 mgr.y (mgr.14146) 165 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:18 smithi155 bash[20805]: audit 2024-02-18T01:14:17.941399+0000 mgr.y (mgr.14146) 166 : audit [DBG] from='client.24233 v1:172.21.15.155:0/580284063' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:14:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:18 smithi155 bash[20805]: audit 2024-02-18T01:14:17.948011+0000 mon.a (mon.0) 369 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:18 smithi155 bash[20805]: audit 2024-02-18T01:14:17.952704+0000 mon.a (mon.0) 370 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:18 smithi155 bash[20805]: audit 2024-02-18T01:14:17.954094+0000 mon.a (mon.0) 371 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: cluster 2024-02-18T01:14:19.505751+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: cephadm 2024-02-18T01:14:19.809567+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:14:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.817870+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.820031+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.821235+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.822281+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.823404+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: cephadm 2024-02-18T01:14:19.824283+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 4073M 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.829527+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.831020+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.832607+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:21.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[19734]: audit 2024-02-18T01:14:19.839645+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: cluster 2024-02-18T01:14:19.505751+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:21.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: cephadm 2024-02-18T01:14:19.809567+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:14:21.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.817870+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.820031+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.821235+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.822281+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.823404+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: cephadm 2024-02-18T01:14:19.824283+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 4073M 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.829527+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.831020+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.832607+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:20 smithi105 bash[27669]: audit 2024-02-18T01:14:19.839645+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: cluster 2024-02-18T01:14:19.505751+0000 mgr.y (mgr.14146) 167 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: cephadm 2024-02-18T01:14:19.809567+0000 mgr.y (mgr.14146) 168 : cephadm [INF] Detected new or changed devices on smithi105 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.817870+0000 mon.a (mon.0) 372 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.820031+0000 mon.a (mon.0) 373 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.821235+0000 mon.a (mon.0) 374 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.822281+0000 mon.a (mon.0) 375 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.823404+0000 mon.a (mon.0) 376 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:14:21.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: cephadm 2024-02-18T01:14:19.824283+0000 mgr.y (mgr.14146) 169 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 4073M 2024-02-18T01:14:21.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.829527+0000 mon.a (mon.0) 377 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:21.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.831020+0000 mon.a (mon.0) 378 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:21.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.832607+0000 mon.a (mon.0) 379 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:21.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:20 smithi155 bash[20805]: audit 2024-02-18T01:14:19.839645+0000 mon.a (mon.0) 380 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:22 smithi105 bash[19734]: cluster 2024-02-18T01:14:21.506317+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:22 smithi105 bash[27669]: cluster 2024-02-18T01:14:21.506317+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:22 smithi155 bash[20805]: cluster 2024-02-18T01:14:21.506317+0000 mgr.y (mgr.14146) 170 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:25.112 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:24 smithi155 bash[20805]: cluster 2024-02-18T01:14:23.506909+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:25.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:24 smithi105 bash[19734]: cluster 2024-02-18T01:14:23.506909+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:24 smithi105 bash[27669]: cluster 2024-02-18T01:14:23.506909+0000 mgr.y (mgr.14146) 171 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:27.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: cluster 2024-02-18T01:14:25.507438+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:27.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: audit 2024-02-18T01:14:25.949022+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.155:0/730207957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: audit 2024-02-18T01:14:25.950151+0000 mon.a (mon.0) 381 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: audit 2024-02-18T01:14:25.959742+0000 mon.a (mon.0) 382 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]': finished 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: cluster 2024-02-18T01:14:25.959998+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: audit 2024-02-18T01:14:25.960181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[19734]: audit 2024-02-18T01:14:26.514998+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.155:0/3605071433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: cluster 2024-02-18T01:14:25.507438+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: audit 2024-02-18T01:14:25.949022+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.155:0/730207957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: audit 2024-02-18T01:14:25.950151+0000 mon.a (mon.0) 381 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: audit 2024-02-18T01:14:25.959742+0000 mon.a (mon.0) 382 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]': finished 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: cluster 2024-02-18T01:14:25.959998+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: audit 2024-02-18T01:14:25.960181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:27.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:26 smithi105 bash[27669]: audit 2024-02-18T01:14:26.514998+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.155:0/3605071433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:14:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: cluster 2024-02-18T01:14:25.507438+0000 mgr.y (mgr.14146) 172 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: audit 2024-02-18T01:14:25.949022+0000 mon.b (mon.1) 3 : audit [INF] from='client.? v1:172.21.15.155:0/730207957' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: audit 2024-02-18T01:14:25.950151+0000 mon.a (mon.0) 381 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]: dispatch 2024-02-18T01:14:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: audit 2024-02-18T01:14:25.959742+0000 mon.a (mon.0) 382 : audit [INF] from='client.24235 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "21226459-02ce-41a2-8761-c8125feea431"}]': finished 2024-02-18T01:14:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: cluster 2024-02-18T01:14:25.959998+0000 mon.a (mon.0) 383 : cluster [DBG] osdmap e27: 5 total, 4 up, 5 in 2024-02-18T01:14:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: audit 2024-02-18T01:14:25.960181+0000 mon.a (mon.0) 384 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:26 smithi155 bash[20805]: audit 2024-02-18T01:14:26.514998+0000 mon.b (mon.1) 4 : audit [DBG] from='client.? v1:172.21.15.155:0/3605071433' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:14:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:28 smithi105 bash[19734]: cluster 2024-02-18T01:14:27.507993+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:28 smithi105 bash[27669]: cluster 2024-02-18T01:14:27.507993+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:28 smithi155 bash[20805]: cluster 2024-02-18T01:14:27.507993+0000 mgr.y (mgr.14146) 173 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:31.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:30 smithi105 bash[19734]: cluster 2024-02-18T01:14:29.508534+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:31.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:30 smithi105 bash[27669]: cluster 2024-02-18T01:14:29.508534+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:30 smithi155 bash[20805]: cluster 2024-02-18T01:14:29.508534+0000 mgr.y (mgr.14146) 174 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:33.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:32 smithi105 bash[19734]: cluster 2024-02-18T01:14:31.509124+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:32 smithi105 bash[27669]: cluster 2024-02-18T01:14:31.509124+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:32 smithi155 bash[20805]: cluster 2024-02-18T01:14:31.509124+0000 mgr.y (mgr.14146) 175 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:35.132 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:34 smithi155 bash[20805]: cluster 2024-02-18T01:14:33.509627+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:35.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:34 smithi105 bash[19734]: cluster 2024-02-18T01:14:33.509627+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:34 smithi105 bash[27669]: cluster 2024-02-18T01:14:33.509627+0000 mgr.y (mgr.14146) 176 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:37.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:36 smithi105 bash[19734]: cluster 2024-02-18T01:14:35.510153+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:37.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:36 smithi105 bash[27669]: cluster 2024-02-18T01:14:35.510153+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:36 smithi155 bash[20805]: cluster 2024-02-18T01:14:35.510153+0000 mgr.y (mgr.14146) 177 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:38.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:37 smithi105 bash[19734]: audit 2024-02-18T01:14:37.606751+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T01:14:38.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:37 smithi105 bash[19734]: audit 2024-02-18T01:14:37.608487+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:38.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:37 smithi105 bash[27669]: audit 2024-02-18T01:14:37.606751+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T01:14:38.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:37 smithi105 bash[27669]: audit 2024-02-18T01:14:37.608487+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:38.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:37 smithi155 bash[20805]: audit 2024-02-18T01:14:37.606751+0000 mon.a (mon.0) 385 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-02-18T01:14:38.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:37 smithi155 bash[20805]: audit 2024-02-18T01:14:37.608487+0000 mon.a (mon.0) 386 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:39.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:38 smithi155 bash[20805]: cluster 2024-02-18T01:14:37.510690+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:39.044 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:38 smithi155 bash[20805]: cephadm 2024-02-18T01:14:37.610133+0000 mgr.y (mgr.14146) 179 : cephadm [INF] Deploying daemon osd.4 on smithi155 2024-02-18T01:14:39.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:38 smithi105 bash[19734]: cluster 2024-02-18T01:14:37.510690+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:39.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:38 smithi105 bash[19734]: cephadm 2024-02-18T01:14:37.610133+0000 mgr.y (mgr.14146) 179 : cephadm [INF] Deploying daemon osd.4 on smithi155 2024-02-18T01:14:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:38 smithi105 bash[27669]: cluster 2024-02-18T01:14:37.510690+0000 mgr.y (mgr.14146) 178 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:38 smithi105 bash[27669]: cephadm 2024-02-18T01:14:37.610133+0000 mgr.y (mgr.14146) 179 : cephadm [INF] Deploying daemon osd.4 on smithi155 2024-02-18T01:14:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:40 smithi155 bash[20805]: cluster 2024-02-18T01:14:39.511255+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:41.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:40 smithi105 bash[19734]: cluster 2024-02-18T01:14:39.511255+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:41.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:40 smithi105 bash[27669]: cluster 2024-02-18T01:14:39.511255+0000 mgr.y (mgr.14146) 180 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:42 smithi155 bash[20805]: audit 2024-02-18T01:14:41.489141+0000 mon.a (mon.0) 387 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:42 smithi155 bash[20805]: cluster 2024-02-18T01:14:41.511778+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:42 smithi155 bash[20805]: audit 2024-02-18T01:14:41.514689+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[19734]: audit 2024-02-18T01:14:41.489141+0000 mon.a (mon.0) 387 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[19734]: cluster 2024-02-18T01:14:41.511778+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[19734]: audit 2024-02-18T01:14:41.514689+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[27669]: audit 2024-02-18T01:14:41.489141+0000 mon.a (mon.0) 387 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[27669]: cluster 2024-02-18T01:14:41.511778+0000 mgr.y (mgr.14146) 181 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:42 smithi105 bash[27669]: audit 2024-02-18T01:14:41.514689+0000 mon.a (mon.0) 388 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:44 smithi105 bash[19734]: cluster 2024-02-18T01:14:43.512300+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:44 smithi105 bash[27669]: cluster 2024-02-18T01:14:43.512300+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:44 smithi155 bash[20805]: cluster 2024-02-18T01:14:43.512300+0000 mgr.y (mgr.14146) 182 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:46.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:46 smithi105 bash[19734]: cluster 2024-02-18T01:14:45.512851+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:46 smithi105 bash[27669]: cluster 2024-02-18T01:14:45.512851+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:46 smithi155 bash[20805]: cluster 2024-02-18T01:14:45.512851+0000 mgr.y (mgr.14146) 183 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:46.739191+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:47.024230+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:47.026168+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:47.035113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:47.527614+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[19734]: audit 2024-02-18T01:14:47.528348+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:46.739191+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:47.024230+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:47.026168+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:47.035113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:47.527614+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:47 smithi105 bash[27669]: audit 2024-02-18T01:14:47.528348+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:46.739191+0000 mon.a (mon.0) 389 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:47.024230+0000 mon.a (mon.0) 390 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:47.026168+0000 mon.a (mon.0) 391 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:14:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:47.035113+0000 mon.a (mon.0) 392 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:47.527614+0000 mon.b (mon.1) 5 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:47 smithi155 bash[20805]: audit 2024-02-18T01:14:47.528348+0000 mon.a (mon.0) 393 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-02-18T01:14:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: cluster 2024-02-18T01:14:47.513401+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: audit 2024-02-18T01:14:48.044807+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: cluster 2024-02-18T01:14:48.045010+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: audit 2024-02-18T01:14:48.045307+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: audit 2024-02-18T01:14:48.049135+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[19734]: audit 2024-02-18T01:14:48.050020+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: cluster 2024-02-18T01:14:47.513401+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: audit 2024-02-18T01:14:48.044807+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T01:14:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: cluster 2024-02-18T01:14:48.045010+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T01:14:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: audit 2024-02-18T01:14:48.045307+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: audit 2024-02-18T01:14:48.049135+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:49 smithi105 bash[27669]: audit 2024-02-18T01:14:48.050020+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: cluster 2024-02-18T01:14:47.513401+0000 mgr.y (mgr.14146) 184 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: audit 2024-02-18T01:14:48.044807+0000 mon.a (mon.0) 394 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: cluster 2024-02-18T01:14:48.045010+0000 mon.a (mon.0) 395 : cluster [DBG] osdmap e28: 5 total, 4 up, 5 in 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: audit 2024-02-18T01:14:48.045307+0000 mon.a (mon.0) 396 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: audit 2024-02-18T01:14:48.049135+0000 mon.b (mon.1) 6 : audit [INF] from='osd.4 v1:172.21.15.155:6800/1608438726' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:49 smithi155 bash[20805]: audit 2024-02-18T01:14:48.050020+0000 mon.a (mon.0) 397 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:14:49.806 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f2339ffb700 1 -- 172.21.15.155:0/580284063 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 3435391443) 0x7f231c000b90 con 0x7f2328062340 2024-02-18T01:14:49.807 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 4 on host 'smithi155' 2024-02-18T01:14:49.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 >> v1:172.21.15.105:6800/3087430119 conn(0x7f2328062340 legacy=0x7f2328064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:49.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 >> v1:172.21.15.105:6790/0 conn(0x7f233c061660 legacy=0x7f233c1079a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:49.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 shutdown_connections 2024-02-18T01:14:49.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 >> 172.21.15.155:0/580284063 conn(0x7f233c0f1eb0 msgr2=0x7f233c101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:14:49.809 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 shutdown_connections 2024-02-18T01:14:49.810 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:49.803+0000 7f23430d7700 1 -- 172.21.15.155:0/580284063 wait complete. 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.047353+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: cluster 2024-02-18T01:14:49.047575+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.049049+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.055003+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: cluster 2024-02-18T01:14:49.065732+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.065895+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.804830+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:50 smithi155 bash[20805]: audit 2024-02-18T01:14:49.811757+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:50.416 DEBUG:teuthology.orchestra.run.smithi155:osd.4> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.4.service 2024-02-18T01:14:50.418 INFO:tasks.cephadm:Deploying osd.5 on smithi155 with /dev/vg_nvme/lv_3... 2024-02-18T01:14:50.418 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-02-18T01:14:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.047353+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: cluster 2024-02-18T01:14:49.047575+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.049049+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.055003+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: cluster 2024-02-18T01:14:49.065732+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.065895+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.804830+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[19734]: audit 2024-02-18T01:14:49.811757+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.047353+0000 mon.a (mon.0) 398 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: cluster 2024-02-18T01:14:49.047575+0000 mon.a (mon.0) 399 : cluster [DBG] osdmap e29: 5 total, 4 up, 5 in 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.049049+0000 mon.a (mon.0) 400 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.055003+0000 mon.a (mon.0) 401 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: cluster 2024-02-18T01:14:49.065732+0000 mon.a (mon.0) 402 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.065895+0000 mon.a (mon.0) 403 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.804830+0000 mon.a (mon.0) 404 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:14:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:50 smithi105 bash[27669]: audit 2024-02-18T01:14:49.811757+0000 mon.a (mon.0) 405 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:14:50.639 INFO:journalctl@ceph.osd.4.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: cluster 2024-02-18T01:14:48.550542+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: cluster 2024-02-18T01:14:48.550656+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: cluster 2024-02-18T01:14:49.514014+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: audit 2024-02-18T01:14:50.058287+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: cluster 2024-02-18T01:14:50.074196+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 v1:172.21.15.155:6800/1608438726 boot 2024-02-18T01:14:51.473 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: cluster 2024-02-18T01:14:50.074291+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T01:14:51.474 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:51 smithi155 bash[20805]: audit 2024-02-18T01:14:50.074485+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:51.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: cluster 2024-02-18T01:14:48.550542+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:51.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: cluster 2024-02-18T01:14:48.550656+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: cluster 2024-02-18T01:14:49.514014+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: audit 2024-02-18T01:14:50.058287+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: cluster 2024-02-18T01:14:50.074196+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 v1:172.21.15.155:6800/1608438726 boot 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: cluster 2024-02-18T01:14:50.074291+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[27669]: audit 2024-02-18T01:14:50.074485+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: cluster 2024-02-18T01:14:48.550542+0000 osd.4 (osd.4) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: cluster 2024-02-18T01:14:48.550656+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: cluster 2024-02-18T01:14:49.514014+0000 mgr.y (mgr.14146) 185 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: audit 2024-02-18T01:14:50.058287+0000 mon.a (mon.0) 406 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: cluster 2024-02-18T01:14:50.074196+0000 mon.a (mon.0) 407 : cluster [INF] osd.4 v1:172.21.15.155:6800/1608438726 boot 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: cluster 2024-02-18T01:14:50.074291+0000 mon.a (mon.0) 408 : cluster [DBG] osdmap e31: 5 total, 5 up, 5 in 2024-02-18T01:14:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:51 smithi105 bash[19734]: audit 2024-02-18T01:14:50.074485+0000 mon.a (mon.0) 409 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:14:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:52 smithi105 bash[27669]: cluster 2024-02-18T01:14:51.075863+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T01:14:52.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:52 smithi105 bash[19734]: cluster 2024-02-18T01:14:51.075863+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T01:14:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:52 smithi155 bash[20805]: cluster 2024-02-18T01:14:51.075863+0000 mon.a (mon.0) 410 : cluster [DBG] osdmap e32: 5 total, 5 up, 5 in 2024-02-18T01:14:53.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:53 smithi105 bash[19734]: cluster 2024-02-18T01:14:51.514586+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:53 smithi105 bash[19734]: cluster 2024-02-18T01:14:52.083400+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T01:14:53.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:53 smithi105 bash[27669]: cluster 2024-02-18T01:14:51.514586+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:53.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:53 smithi105 bash[27669]: cluster 2024-02-18T01:14:52.083400+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T01:14:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:53 smithi155 bash[20805]: cluster 2024-02-18T01:14:51.514586+0000 mgr.y (mgr.14146) 186 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:53 smithi155 bash[20805]: cluster 2024-02-18T01:14:52.083400+0000 mon.a (mon.0) 411 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2024-02-18T01:14:53.636 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-02-18T01:14:53.636 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-02-18T01:14:53.636 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-18T01:14:53.637 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-18T01:14:53.637 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0163814 s, 640 MB/s 2024-02-18T01:14:53.637 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-18T01:14:54.228 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_3 2024-02-18T01:14:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:55 smithi105 bash[19734]: cluster 2024-02-18T01:14:53.515159+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:55 smithi105 bash[27669]: cluster 2024-02-18T01:14:53.515159+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:55 smithi155 bash[20805]: cluster 2024-02-18T01:14:53.515159+0000 mgr.y (mgr.14146) 187 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:57.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:57 smithi105 bash[19734]: cluster 2024-02-18T01:14:55.515678+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:57.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:57 smithi105 bash[27669]: cluster 2024-02-18T01:14:55.515678+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:57.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:57 smithi155 bash[20805]: cluster 2024-02-18T01:14:55.515678+0000 mgr.y (mgr.14146) 188 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1666078617 >> v1:172.21.15.155:6789/0 conn(0x7f6e480fbbb0 legacy=0x7f6e480fe050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1666078617 shutdown_connections 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1666078617 >> 172.21.15.155:0/1666078617 conn(0x7f6e480f1e60 msgr2=0x7f6e480f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1666078617 shutdown_connections 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1666078617 wait complete. 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 Processor -- start 2024-02-18T01:14:58.405 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- start start 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6e48114d10 con 0x7f6e48061240 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6e48115ef0 con 0x7f6e480ff910 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4ea29700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6e481170f0 con 0x7f6e480fbbb0 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4cfc6700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f6e480ff910 0x7f6e481133d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:52528/0 (socket says 172.21.15.155:52528) 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.398+0000 7f6e4cfc6700 1 -- 172.21.15.155:0/1480324313 learned_addr learned my addr 172.21.15.155:0/1480324313 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4047423613 0 0) 0x7f6e48115ef0 con 0x7f6e480ff910 2024-02-18T01:14:58.407 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e34003030 con 0x7f6e480ff910 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555062612 0 0) 0x7f6e481170f0 con 0x7f6e480fbbb0 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e48115ef0 con 0x7f6e480fbbb0 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4213864259 0 0) 0x7f6e48114d10 con 0x7f6e48061240 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e481170f0 con 0x7f6e48061240 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3635712488 0 0) 0x7f6e34003030 con 0x7f6e480ff910 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e48114d10 con 0x7f6e480ff910 2024-02-18T01:14:58.408 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6e38001900 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3058248892 0 0) 0x7f6e48115ef0 con 0x7f6e480fbbb0 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6e34003030 con 0x7f6e480fbbb0 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6e30001c40 con 0x7f6e480fbbb0 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1863337493 0 0) 0x7f6e48114d10 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 >> v1:172.21.15.105:6790/0 conn(0x7f6e480fbbb0 legacy=0x7f6e4810fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 >> v1:172.21.15.105:6789/0 conn(0x7f6e48061240 legacy=0x7f6e480f7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e481182f0 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e481172e0 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e48117750 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f6e38001c90 con 0x7f6e480ff910 2024-02-18T01:14:58.409 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6e38005ae0 con 0x7f6e480ff910 2024-02-18T01:14:58.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28004f20 con 0x7f6e480ff910 2024-02-18T01:14:58.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f6e380021b0 con 0x7f6e480ff910 2024-02-18T01:14:58.410 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.402+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(33..33 src has 1..33) v4 ==== 3178+0+0 (unknown 3036696457 0 0) 0x7f6e380778d0 con 0x7f6e480ff910 2024-02-18T01:14:58.417 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.410+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f6e38048510 con 0x7f6e480ff910 2024-02-18T01:14:58.613 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:14:58.606+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e28000b90 con 0x7f6e34062150 2024-02-18T01:14:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[19734]: cluster 2024-02-18T01:14:57.516310+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[19734]: audit 2024-02-18T01:14:58.621966+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[19734]: audit 2024-02-18T01:14:58.628248+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[19734]: audit 2024-02-18T01:14:58.629764+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[27669]: cluster 2024-02-18T01:14:57.516310+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[27669]: audit 2024-02-18T01:14:58.621966+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[27669]: audit 2024-02-18T01:14:58.628248+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:14:59 smithi105 bash[27669]: audit 2024-02-18T01:14:58.629764+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:14:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:59 smithi155 bash[20805]: cluster 2024-02-18T01:14:57.516310+0000 mgr.y (mgr.14146) 189 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:14:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:59 smithi155 bash[20805]: audit 2024-02-18T01:14:58.621966+0000 mon.a (mon.0) 412 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:14:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:59 smithi155 bash[20805]: audit 2024-02-18T01:14:58.628248+0000 mon.a (mon.0) 413 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:14:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:14:59 smithi155 bash[20805]: audit 2024-02-18T01:14:58.629764+0000 mon.a (mon.0) 414 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:58.614926+0000 mgr.y (mgr.14146) 190 : audit [DBG] from='client.24256 v1:172.21.15.155:0/1480324313' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.827450+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.829715+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.840485+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.842473+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.843988+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[19734]: audit 2024-02-18T01:14:59.854257+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:58.614926+0000 mgr.y (mgr.14146) 190 : audit [DBG] from='client.24256 v1:172.21.15.155:0/1480324313' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.827450+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.829715+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.840485+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.842473+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.843988+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:00 smithi105 bash[27669]: audit 2024-02-18T01:14:59.854257+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:58.614926+0000 mgr.y (mgr.14146) 190 : audit [DBG] from='client.24256 v1:172.21.15.155:0/1480324313' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.827450+0000 mon.a (mon.0) 415 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.829715+0000 mon.a (mon.0) 416 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.840485+0000 mon.a (mon.0) 417 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.842473+0000 mon.a (mon.0) 418 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.843988+0000 mon.a (mon.0) 419 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:00 smithi155 bash[20805]: audit 2024-02-18T01:14:59.854257+0000 mon.a (mon.0) 420 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[19734]: cluster 2024-02-18T01:14:59.516886+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[19734]: cephadm 2024-02-18T01:14:59.818024+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[19734]: cephadm 2024-02-18T01:14:59.831195+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 17318M 2024-02-18T01:15:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[27669]: cluster 2024-02-18T01:14:59.516886+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[27669]: cephadm 2024-02-18T01:14:59.818024+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:01 smithi105 bash[27669]: cephadm 2024-02-18T01:14:59.831195+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 17318M 2024-02-18T01:15:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:01 smithi155 bash[20805]: cluster 2024-02-18T01:14:59.516886+0000 mgr.y (mgr.14146) 191 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:01 smithi155 bash[20805]: cephadm 2024-02-18T01:14:59.818024+0000 mgr.y (mgr.14146) 192 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:01 smithi155 bash[20805]: cephadm 2024-02-18T01:14:59.831195+0000 mgr.y (mgr.14146) 193 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 17318M 2024-02-18T01:15:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:03 smithi105 bash[19734]: cluster 2024-02-18T01:15:01.517494+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:03 smithi105 bash[27669]: cluster 2024-02-18T01:15:01.517494+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:03 smithi155 bash[20805]: cluster 2024-02-18T01:15:01.517494+0000 mgr.y (mgr.14146) 194 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:05 smithi105 bash[19734]: cluster 2024-02-18T01:15:03.518126+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:05 smithi105 bash[27669]: cluster 2024-02-18T01:15:03.518126+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:05 smithi155 bash[20805]: cluster 2024-02-18T01:15:03.518126+0000 mgr.y (mgr.14146) 195 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: cluster 2024-02-18T01:15:05.518696+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: audit 2024-02-18T01:15:06.624238+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.155:0/3200076485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: audit 2024-02-18T01:15:06.625501+0000 mon.a (mon.0) 421 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: audit 2024-02-18T01:15:06.634330+0000 mon.a (mon.0) 422 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]': finished 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: cluster 2024-02-18T01:15:06.634532+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[19734]: audit 2024-02-18T01:15:06.634735+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: cluster 2024-02-18T01:15:05.518696+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: audit 2024-02-18T01:15:06.624238+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.155:0/3200076485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: audit 2024-02-18T01:15:06.625501+0000 mon.a (mon.0) 421 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: audit 2024-02-18T01:15:06.634330+0000 mon.a (mon.0) 422 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]': finished 2024-02-18T01:15:07.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: cluster 2024-02-18T01:15:06.634532+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-18T01:15:07.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:07 smithi105 bash[27669]: audit 2024-02-18T01:15:06.634735+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: cluster 2024-02-18T01:15:05.518696+0000 mgr.y (mgr.14146) 196 : cluster [DBG] pgmap v178: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: audit 2024-02-18T01:15:06.624238+0000 mon.b (mon.1) 7 : audit [INF] from='client.? v1:172.21.15.155:0/3200076485' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: audit 2024-02-18T01:15:06.625501+0000 mon.a (mon.0) 421 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]: dispatch 2024-02-18T01:15:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: audit 2024-02-18T01:15:06.634330+0000 mon.a (mon.0) 422 : audit [INF] from='client.24262 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2650bfc0-0a69-4425-aaef-31bada765f29"}]': finished 2024-02-18T01:15:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: cluster 2024-02-18T01:15:06.634532+0000 mon.a (mon.0) 423 : cluster [DBG] osdmap e34: 6 total, 5 up, 6 in 2024-02-18T01:15:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:07 smithi155 bash[20805]: audit 2024-02-18T01:15:06.634735+0000 mon.a (mon.0) 424 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:08.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:08 smithi105 bash[19734]: audit 2024-02-18T01:15:07.233773+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.155:0/1496612891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:08.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:08 smithi105 bash[27669]: audit 2024-02-18T01:15:07.233773+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.155:0/1496612891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:08 smithi155 bash[20805]: audit 2024-02-18T01:15:07.233773+0000 mon.b (mon.1) 8 : audit [DBG] from='client.? v1:172.21.15.155:0/1496612891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:09 smithi105 bash[19734]: cluster 2024-02-18T01:15:07.519330+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:09 smithi105 bash[27669]: cluster 2024-02-18T01:15:07.519330+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:09 smithi155 bash[20805]: cluster 2024-02-18T01:15:07.519330+0000 mgr.y (mgr.14146) 197 : cluster [DBG] pgmap v180: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:11 smithi105 bash[19734]: cluster 2024-02-18T01:15:09.519901+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:11 smithi105 bash[27669]: cluster 2024-02-18T01:15:09.519901+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:11 smithi155 bash[20805]: cluster 2024-02-18T01:15:09.519901+0000 mgr.y (mgr.14146) 198 : cluster [DBG] pgmap v181: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:12 smithi105 bash[19734]: cluster 2024-02-18T01:15:11.520474+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:12.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:12 smithi105 bash[27669]: cluster 2024-02-18T01:15:11.520474+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:12 smithi155 bash[20805]: cluster 2024-02-18T01:15:11.520474+0000 mgr.y (mgr.14146) 199 : cluster [DBG] pgmap v182: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:14 smithi105 bash[19734]: cluster 2024-02-18T01:15:13.521025+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:14.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:14 smithi105 bash[27669]: cluster 2024-02-18T01:15:13.521025+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:14 smithi155 bash[20805]: cluster 2024-02-18T01:15:13.521025+0000 mgr.y (mgr.14146) 200 : cluster [DBG] pgmap v183: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:16.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:16 smithi105 bash[19734]: cluster 2024-02-18T01:15:15.521541+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:16.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:16 smithi105 bash[27669]: cluster 2024-02-18T01:15:15.521541+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:16 smithi155 bash[20805]: cluster 2024-02-18T01:15:15.521541+0000 mgr.y (mgr.14146) 201 : cluster [DBG] pgmap v184: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:18.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:18 smithi105 bash[27669]: cluster 2024-02-18T01:15:17.522035+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:18 smithi105 bash[19734]: cluster 2024-02-18T01:15:17.522035+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:18 smithi155 bash[20805]: cluster 2024-02-18T01:15:17.522035+0000 mgr.y (mgr.14146) 202 : cluster [DBG] pgmap v185: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:19.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[27669]: audit 2024-02-18T01:15:18.624979+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T01:15:19.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[27669]: audit 2024-02-18T01:15:18.626925+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[27669]: cephadm 2024-02-18T01:15:18.628600+0000 mgr.y (mgr.14146) 203 : cephadm [INF] Deploying daemon osd.5 on smithi155 2024-02-18T01:15:19.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[19734]: audit 2024-02-18T01:15:18.624979+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T01:15:19.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[19734]: audit 2024-02-18T01:15:18.626925+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:19.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:19 smithi105 bash[19734]: cephadm 2024-02-18T01:15:18.628600+0000 mgr.y (mgr.14146) 203 : cephadm [INF] Deploying daemon osd.5 on smithi155 2024-02-18T01:15:19.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:19 smithi155 bash[20805]: audit 2024-02-18T01:15:18.624979+0000 mon.a (mon.0) 425 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-02-18T01:15:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:19 smithi155 bash[20805]: audit 2024-02-18T01:15:18.626925+0000 mon.a (mon.0) 426 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:19 smithi155 bash[20805]: cephadm 2024-02-18T01:15:18.628600+0000 mgr.y (mgr.14146) 203 : cephadm [INF] Deploying daemon osd.5 on smithi155 2024-02-18T01:15:20.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:20 smithi105 bash[19734]: cluster 2024-02-18T01:15:19.522580+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:20 smithi105 bash[27669]: cluster 2024-02-18T01:15:19.522580+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:20.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:20 smithi155 bash[20805]: cluster 2024-02-18T01:15:19.522580+0000 mgr.y (mgr.14146) 204 : cluster [DBG] pgmap v186: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:22.658 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:22 smithi155 bash[20805]: cluster 2024-02-18T01:15:21.523155+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:22.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:22 smithi105 bash[19734]: cluster 2024-02-18T01:15:21.523155+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:22.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:22 smithi105 bash[27669]: cluster 2024-02-18T01:15:21.523155+0000 mgr.y (mgr.14146) 205 : cluster [DBG] pgmap v187: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:23.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:23 smithi105 bash[19734]: audit 2024-02-18T01:15:22.701105+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:23.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:23 smithi105 bash[19734]: audit 2024-02-18T01:15:22.732255+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:23 smithi105 bash[27669]: audit 2024-02-18T01:15:22.701105+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:23 smithi105 bash[27669]: audit 2024-02-18T01:15:22.732255+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:23.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:23 smithi155 bash[20805]: audit 2024-02-18T01:15:22.701105+0000 mon.a (mon.0) 427 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:23.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:23 smithi155 bash[20805]: audit 2024-02-18T01:15:22.732255+0000 mon.a (mon.0) 428 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:24.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:24 smithi105 bash[19734]: cluster 2024-02-18T01:15:23.523721+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:24.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:24 smithi105 bash[27669]: cluster 2024-02-18T01:15:23.523721+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:24.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:24 smithi155 bash[20805]: cluster 2024-02-18T01:15:23.523721+0000 mgr.y (mgr.14146) 206 : cluster [DBG] pgmap v188: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:26.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:26 smithi105 bash[19734]: cluster 2024-02-18T01:15:25.524263+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:26.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:26 smithi105 bash[27669]: cluster 2024-02-18T01:15:25.524263+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:26.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:26 smithi155 bash[20805]: cluster 2024-02-18T01:15:25.524263+0000 mgr.y (mgr.14146) 207 : cluster [DBG] pgmap v189: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:29.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:29 smithi155 bash[20805]: cluster 2024-02-18T01:15:27.524941+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:29.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:29 smithi155 bash[20805]: audit 2024-02-18T01:15:28.018746+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:29.274 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:29 smithi155 bash[20805]: audit 2024-02-18T01:15:28.020448+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:29.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:29 smithi155 bash[20805]: audit 2024-02-18T01:15:28.022146+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:29.275 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:29 smithi155 bash[20805]: audit 2024-02-18T01:15:28.033478+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[19734]: cluster 2024-02-18T01:15:27.524941+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[19734]: audit 2024-02-18T01:15:28.018746+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[19734]: audit 2024-02-18T01:15:28.020448+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[19734]: audit 2024-02-18T01:15:28.022146+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[19734]: audit 2024-02-18T01:15:28.033478+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[27669]: cluster 2024-02-18T01:15:27.524941+0000 mgr.y (mgr.14146) 208 : cluster [DBG] pgmap v190: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[27669]: audit 2024-02-18T01:15:28.018746+0000 mon.a (mon.0) 429 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[27669]: audit 2024-02-18T01:15:28.020448+0000 mon.a (mon.0) 430 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[27669]: audit 2024-02-18T01:15:28.022146+0000 mon.a (mon.0) 431 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:29 smithi105 bash[27669]: audit 2024-02-18T01:15:28.033478+0000 mon.a (mon.0) 432 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:30.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:30 smithi105 bash[19734]: audit 2024-02-18T01:15:29.277522+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:30.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:30 smithi105 bash[19734]: audit 2024-02-18T01:15:29.278345+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:30.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:30 smithi105 bash[27669]: audit 2024-02-18T01:15:29.277522+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:30.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:30 smithi105 bash[27669]: audit 2024-02-18T01:15:29.278345+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:30.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:30 smithi155 bash[20805]: audit 2024-02-18T01:15:29.277522+0000 mon.b (mon.1) 9 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:30.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:30 smithi155 bash[20805]: audit 2024-02-18T01:15:29.278345+0000 mon.a (mon.0) 433 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-02-18T01:15:31.142 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 5 on host 'smithi155' 2024-02-18T01:15:31.143 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e457fa700 1 -- 172.21.15.155:0/1480324313 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 2429945703) 0x7f6e28000b90 con 0x7f6e34062150 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 >> v1:172.21.15.105:6800/3087430119 conn(0x7f6e34062150 legacy=0x7f6e34064610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 >> v1:172.21.15.155:6789/0 conn(0x7f6e480ff910 legacy=0x7f6e481133d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 shutdown_connections 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 >> 172.21.15.155:0/1480324313 conn(0x7f6e480f1e60 msgr2=0x7f6e48101dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 shutdown_connections 2024-02-18T01:15:31.145 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:31.138+0000 7f6e4ea29700 1 -- 172.21.15.155:0/1480324313 wait complete. 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: cluster 2024-02-18T01:15:29.525544+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: audit 2024-02-18T01:15:30.036263+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: cluster 2024-02-18T01:15:30.036359+0000 mon.a (mon.0) 435 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: audit 2024-02-18T01:15:30.036575+0000 mon.a (mon.0) 436 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: audit 2024-02-18T01:15:30.037191+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[19734]: audit 2024-02-18T01:15:30.037889+0000 mon.a (mon.0) 437 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: cluster 2024-02-18T01:15:29.525544+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:31.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: audit 2024-02-18T01:15:30.036263+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T01:15:31.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: cluster 2024-02-18T01:15:30.036359+0000 mon.a (mon.0) 435 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T01:15:31.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: audit 2024-02-18T01:15:30.036575+0000 mon.a (mon.0) 436 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:31.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: audit 2024-02-18T01:15:30.037191+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:31 smithi105 bash[27669]: audit 2024-02-18T01:15:30.037889+0000 mon.a (mon.0) 437 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: cluster 2024-02-18T01:15:29.525544+0000 mgr.y (mgr.14146) 209 : cluster [DBG] pgmap v191: 1 pgs: 1 active+clean; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: audit 2024-02-18T01:15:30.036263+0000 mon.a (mon.0) 434 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-02-18T01:15:31.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: cluster 2024-02-18T01:15:30.036359+0000 mon.a (mon.0) 435 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2024-02-18T01:15:31.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: audit 2024-02-18T01:15:30.036575+0000 mon.a (mon.0) 436 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:31.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: audit 2024-02-18T01:15:30.037191+0000 mon.b (mon.1) 10 : audit [INF] from='osd.5 v1:172.21.15.155:6804/3674587623' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:31 smithi155 bash[20805]: audit 2024-02-18T01:15:30.037889+0000 mon.a (mon.0) 437 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:15:31.860 DEBUG:teuthology.orchestra.run.smithi155:osd.5> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.5.service 2024-02-18T01:15:31.863 INFO:tasks.cephadm:Deploying osd.6 on smithi155 with /dev/vg_nvme/lv_2... 2024-02-18T01:15:31.863 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-02-18T01:15:32.013 INFO:journalctl@ceph.osd.5.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:15:32.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: audit 2024-02-18T01:15:31.038621+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:15:32.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: cluster 2024-02-18T01:15:31.038701+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T01:15:32.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: audit 2024-02-18T01:15:31.039389+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: audit 2024-02-18T01:15:31.044530+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: audit 2024-02-18T01:15:31.140598+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:32.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:32 smithi155 bash[20805]: audit 2024-02-18T01:15:31.159657+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: audit 2024-02-18T01:15:31.038621+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: cluster 2024-02-18T01:15:31.038701+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: audit 2024-02-18T01:15:31.039389+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: audit 2024-02-18T01:15:31.044530+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: audit 2024-02-18T01:15:31.140598+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[19734]: audit 2024-02-18T01:15:31.159657+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: audit 2024-02-18T01:15:31.038621+0000 mon.a (mon.0) 438 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: cluster 2024-02-18T01:15:31.038701+0000 mon.a (mon.0) 439 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2024-02-18T01:15:32.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: audit 2024-02-18T01:15:31.039389+0000 mon.a (mon.0) 440 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: audit 2024-02-18T01:15:31.044530+0000 mon.a (mon.0) 441 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:32.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: audit 2024-02-18T01:15:31.140598+0000 mon.a (mon.0) 442 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:32.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:32 smithi105 bash[27669]: audit 2024-02-18T01:15:31.159657+0000 mon.a (mon.0) 443 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: cluster 2024-02-18T01:15:30.304937+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: cluster 2024-02-18T01:15:30.304998+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: cluster 2024-02-18T01:15:31.526104+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: audit 2024-02-18T01:15:32.048457+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: cluster 2024-02-18T01:15:32.054796+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 v1:172.21.15.155:6804/3674587623 boot 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: cluster 2024-02-18T01:15:32.054900+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-18T01:15:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:33 smithi155 bash[20805]: audit 2024-02-18T01:15:32.055161+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:33.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: cluster 2024-02-18T01:15:30.304937+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: cluster 2024-02-18T01:15:30.304998+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: cluster 2024-02-18T01:15:31.526104+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: audit 2024-02-18T01:15:32.048457+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: cluster 2024-02-18T01:15:32.054796+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 v1:172.21.15.155:6804/3674587623 boot 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: cluster 2024-02-18T01:15:32.054900+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[19734]: audit 2024-02-18T01:15:32.055161+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: cluster 2024-02-18T01:15:30.304937+0000 osd.5 (osd.5) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: cluster 2024-02-18T01:15:30.304998+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: cluster 2024-02-18T01:15:31.526104+0000 mgr.y (mgr.14146) 210 : cluster [DBG] pgmap v194: 1 pgs: 1 peering; 0 B data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-02-18T01:15:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: audit 2024-02-18T01:15:32.048457+0000 mon.a (mon.0) 444 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: cluster 2024-02-18T01:15:32.054796+0000 mon.a (mon.0) 445 : cluster [INF] osd.5 v1:172.21.15.155:6804/3674587623 boot 2024-02-18T01:15:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: cluster 2024-02-18T01:15:32.054900+0000 mon.a (mon.0) 446 : cluster [DBG] osdmap e37: 6 total, 6 up, 6 in 2024-02-18T01:15:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:33 smithi105 bash[27669]: audit 2024-02-18T01:15:32.055161+0000 mon.a (mon.0) 447 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:15:34.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:34 smithi105 bash[19734]: cluster 2024-02-18T01:15:33.060465+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-18T01:15:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:34 smithi105 bash[27669]: cluster 2024-02-18T01:15:33.060465+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-18T01:15:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:34 smithi155 bash[20805]: cluster 2024-02-18T01:15:33.060465+0000 mon.a (mon.0) 448 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0171814 s, 610 MB/s 2024-02-18T01:15:35.094 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-18T01:15:35.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:35 smithi105 bash[19734]: cluster 2024-02-18T01:15:33.526713+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v197: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:35.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:35 smithi105 bash[19734]: cluster 2024-02-18T01:15:34.070377+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T01:15:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:35 smithi105 bash[27669]: cluster 2024-02-18T01:15:33.526713+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v197: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:35 smithi105 bash[27669]: cluster 2024-02-18T01:15:34.070377+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T01:15:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:35 smithi155 bash[20805]: cluster 2024-02-18T01:15:33.526713+0000 mgr.y (mgr.14146) 211 : cluster [DBG] pgmap v197: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:35 smithi155 bash[20805]: cluster 2024-02-18T01:15:34.070377+0000 mon.a (mon.0) 449 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2024-02-18T01:15:35.796 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_2 2024-02-18T01:15:37.330 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:37 smithi155 bash[20805]: cluster 2024-02-18T01:15:35.527274+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:37 smithi105 bash[19734]: cluster 2024-02-18T01:15:35.527274+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:37 smithi105 bash[27669]: cluster 2024-02-18T01:15:35.527274+0000 mgr.y (mgr.14146) 212 : cluster [DBG] pgmap v199: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:39.474 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:39 smithi155 bash[20805]: cluster 2024-02-18T01:15:37.527826+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:39 smithi105 bash[19734]: cluster 2024-02-18T01:15:37.527826+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:39 smithi105 bash[27669]: cluster 2024-02-18T01:15:37.527826+0000 mgr.y (mgr.14146) 213 : cluster [DBG] pgmap v200: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:39.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/22756794 >> v1:172.21.15.155:6789/0 conn(0x7fa8740f6720 legacy=0x7fa8740f6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:15:39.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/22756794 shutdown_connections 2024-02-18T01:15:39.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/22756794 >> 172.21.15.155:0/22756794 conn(0x7fa8740f1e60 msgr2=0x7fa8740f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:15:39.716 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/22756794 shutdown_connections 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/22756794 wait complete. 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 Processor -- start 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- start start 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa874112c70 con 0x7fa8740f6720 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa874113e10 con 0x7fa8740fe820 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa874114fb0 con 0x7fa8740fabf0 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa8737fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fa8740fe820 0x7fa874111550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:50366/0 (socket says 172.21.15.155:50366) 2024-02-18T01:15:39.717 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa8737fe700 1 -- 172.21.15.155:0/1899383558 learned_addr learned my addr 172.21.15.155:0/1899383558 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4240365820 0 0) 0x7fa874113e10 con 0x7fa8740fe820 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85c003030 con 0x7fa8740fe820 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2529699819 0 0) 0x7fa874112c70 con 0x7fa8740f6720 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa874113e10 con 0x7fa8740f6720 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2535089365 0 0) 0x7fa874114fb0 con 0x7fa8740fabf0 2024-02-18T01:15:39.724 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa874112c70 con 0x7fa8740fabf0 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1825520640 0 0) 0x7fa85c003030 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa874114fb0 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa86c001c40 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2011490528 0 0) 0x7fa874113e10 con 0x7fa8740f6720 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa85c003030 con 0x7fa8740f6720 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa864001510 con 0x7fa8740f6720 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3579287829 0 0) 0x7fa874114fb0 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 >> v1:172.21.15.105:6790/0 conn(0x7fa8740fabf0 legacy=0x7fa87410de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 >> v1:172.21.15.105:6789/0 conn(0x7fa8740f6720 legacy=0x7fa87410a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa874116150 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa874112e60 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.709+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa874113350 con 0x7fa8740fe820 2024-02-18T01:15:39.725 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.713+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fa86c001fd0 con 0x7fa8740fe820 2024-02-18T01:15:39.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.713+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa86c005e10 con 0x7fa8740fe820 2024-02-18T01:15:39.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.713+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa860004f20 con 0x7fa8740fe820 2024-02-18T01:15:39.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.713+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7fa86c002530 con 0x7fa8740fe820 2024-02-18T01:15:39.726 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.713+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(39..39 src has 1..39) v4 ==== 3470+0+0 (unknown 792560735 0 0) 0x7fa86c077d50 con 0x7fa8740fe820 2024-02-18T01:15:39.727 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.721+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa86c048810 con 0x7fa8740fe820 2024-02-18T01:15:39.918 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:15:39.913+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa860000b90 con 0x7fa85c0621a0 2024-02-18T01:15:40.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[19734]: audit 2024-02-18T01:15:39.926968+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:15:40.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[19734]: audit 2024-02-18T01:15:39.932996+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:15:40.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[19734]: audit 2024-02-18T01:15:39.934133+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:40.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[27669]: audit 2024-02-18T01:15:39.926968+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:15:40.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[27669]: audit 2024-02-18T01:15:39.932996+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:15:40.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:40 smithi105 bash[27669]: audit 2024-02-18T01:15:39.934133+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:40 smithi155 bash[20805]: audit 2024-02-18T01:15:39.926968+0000 mon.a (mon.0) 450 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:15:40.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:40 smithi155 bash[20805]: audit 2024-02-18T01:15:39.932996+0000 mon.a (mon.0) 451 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:15:40.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:40 smithi155 bash[20805]: audit 2024-02-18T01:15:39.934133+0000 mon.a (mon.0) 452 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:41 smithi105 bash[19734]: cluster 2024-02-18T01:15:39.528439+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:41 smithi105 bash[19734]: audit 2024-02-18T01:15:39.919960+0000 mgr.y (mgr.14146) 215 : audit [DBG] from='client.24283 v1:172.21.15.155:0/1899383558' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:41.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:41 smithi105 bash[27669]: cluster 2024-02-18T01:15:39.528439+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:41.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:41 smithi105 bash[27669]: audit 2024-02-18T01:15:39.919960+0000 mgr.y (mgr.14146) 215 : audit [DBG] from='client.24283 v1:172.21.15.155:0/1899383558' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:41 smithi155 bash[20805]: cluster 2024-02-18T01:15:39.528439+0000 mgr.y (mgr.14146) 214 : cluster [DBG] pgmap v201: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:41 smithi155 bash[20805]: audit 2024-02-18T01:15:39.919960+0000 mgr.y (mgr.14146) 215 : audit [DBG] from='client.24283 v1:172.21.15.155:0/1899383558' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:15:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: cephadm 2024-02-18T01:15:41.143218+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.153092+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.155354+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.157305+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.167541+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.169282+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.170914+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[19734]: audit 2024-02-18T01:15:41.181517+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: cephadm 2024-02-18T01:15:41.143218+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.153092+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.155354+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.157305+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.167541+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.169282+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.170914+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:42.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:42 smithi105 bash[27669]: audit 2024-02-18T01:15:41.181517+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: cephadm 2024-02-18T01:15:41.143218+0000 mgr.y (mgr.14146) 216 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.153092+0000 mon.a (mon.0) 453 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.155354+0000 mon.a (mon.0) 454 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.157305+0000 mon.a (mon.0) 455 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.167541+0000 mon.a (mon.0) 456 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.169282+0000 mon.a (mon.0) 457 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.170914+0000 mon.a (mon.0) 458 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:15:42.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:42 smithi155 bash[20805]: audit 2024-02-18T01:15:41.181517+0000 mon.a (mon.0) 459 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:15:43.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:43 smithi105 bash[19734]: cephadm 2024-02-18T01:15:41.158669+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 8659M 2024-02-18T01:15:43.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:43 smithi105 bash[19734]: cluster 2024-02-18T01:15:41.529060+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:43.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:43 smithi105 bash[27669]: cephadm 2024-02-18T01:15:41.158669+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 8659M 2024-02-18T01:15:43.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:43 smithi105 bash[27669]: cluster 2024-02-18T01:15:41.529060+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:43 smithi155 bash[20805]: cephadm 2024-02-18T01:15:41.158669+0000 mgr.y (mgr.14146) 217 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 8659M 2024-02-18T01:15:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:43 smithi155 bash[20805]: cluster 2024-02-18T01:15:41.529060+0000 mgr.y (mgr.14146) 218 : cluster [DBG] pgmap v202: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:45 smithi105 bash[19734]: cluster 2024-02-18T01:15:43.529727+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:45 smithi105 bash[27669]: cluster 2024-02-18T01:15:43.529727+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:45 smithi155 bash[20805]: cluster 2024-02-18T01:15:43.529727+0000 mgr.y (mgr.14146) 219 : cluster [DBG] pgmap v203: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:47.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:47 smithi105 bash[19734]: cluster 2024-02-18T01:15:45.530385+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:47 smithi105 bash[27669]: cluster 2024-02-18T01:15:45.530385+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:47 smithi155 bash[20805]: cluster 2024-02-18T01:15:45.530385+0000 mgr.y (mgr.14146) 220 : cluster [DBG] pgmap v204: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[19734]: audit 2024-02-18T01:15:48.081279+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.155:0/1723621399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[19734]: audit 2024-02-18T01:15:48.082303+0000 mon.a (mon.0) 460 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[19734]: audit 2024-02-18T01:15:48.090074+0000 mon.a (mon.0) 461 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]': finished 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[19734]: cluster 2024-02-18T01:15:48.090146+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[19734]: audit 2024-02-18T01:15:48.090482+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[27669]: audit 2024-02-18T01:15:48.081279+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.155:0/1723621399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[27669]: audit 2024-02-18T01:15:48.082303+0000 mon.a (mon.0) 460 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[27669]: audit 2024-02-18T01:15:48.090074+0000 mon.a (mon.0) 461 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]': finished 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[27669]: cluster 2024-02-18T01:15:48.090146+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-18T01:15:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:48 smithi105 bash[27669]: audit 2024-02-18T01:15:48.090482+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:15:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:48 smithi155 bash[20805]: audit 2024-02-18T01:15:48.081279+0000 mon.b (mon.1) 11 : audit [INF] from='client.? v1:172.21.15.155:0/1723621399' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:48 smithi155 bash[20805]: audit 2024-02-18T01:15:48.082303+0000 mon.a (mon.0) 460 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]: dispatch 2024-02-18T01:15:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:48 smithi155 bash[20805]: audit 2024-02-18T01:15:48.090074+0000 mon.a (mon.0) 461 : audit [INF] from='client.24289 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "565ca7a5-3796-4f8a-93be-8e98f818ff85"}]': finished 2024-02-18T01:15:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:48 smithi155 bash[20805]: cluster 2024-02-18T01:15:48.090146+0000 mon.a (mon.0) 462 : cluster [DBG] osdmap e40: 7 total, 6 up, 7 in 2024-02-18T01:15:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:48 smithi155 bash[20805]: audit 2024-02-18T01:15:48.090482+0000 mon.a (mon.0) 463 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:15:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:49 smithi105 bash[19734]: cluster 2024-02-18T01:15:47.530973+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:49 smithi105 bash[19734]: audit 2024-02-18T01:15:48.687559+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.155:0/3457299701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:49 smithi105 bash[27669]: cluster 2024-02-18T01:15:47.530973+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:49 smithi105 bash[27669]: audit 2024-02-18T01:15:48.687559+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.155:0/3457299701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:49 smithi155 bash[20805]: cluster 2024-02-18T01:15:47.530973+0000 mgr.y (mgr.14146) 221 : cluster [DBG] pgmap v205: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:49 smithi155 bash[20805]: audit 2024-02-18T01:15:48.687559+0000 mon.b (mon.1) 12 : audit [DBG] from='client.? v1:172.21.15.155:0/3457299701' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:15:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:51 smithi105 bash[19734]: cluster 2024-02-18T01:15:49.531550+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:51 smithi105 bash[27669]: cluster 2024-02-18T01:15:49.531550+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:51 smithi155 bash[20805]: cluster 2024-02-18T01:15:49.531550+0000 mgr.y (mgr.14146) 222 : cluster [DBG] pgmap v207: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:53.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:53 smithi105 bash[27669]: cluster 2024-02-18T01:15:51.532143+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:53 smithi105 bash[19734]: cluster 2024-02-18T01:15:51.532143+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:53 smithi155 bash[20805]: cluster 2024-02-18T01:15:51.532143+0000 mgr.y (mgr.14146) 223 : cluster [DBG] pgmap v208: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:55.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:55 smithi105 bash[19734]: cluster 2024-02-18T01:15:53.532756+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:55 smithi105 bash[27669]: cluster 2024-02-18T01:15:53.532756+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:55 smithi155 bash[20805]: cluster 2024-02-18T01:15:53.532756+0000 mgr.y (mgr.14146) 224 : cluster [DBG] pgmap v209: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:57.444 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:57 smithi155 bash[20805]: cluster 2024-02-18T01:15:55.533323+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:57.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:57 smithi105 bash[19734]: cluster 2024-02-18T01:15:55.533323+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:57.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:57 smithi105 bash[27669]: cluster 2024-02-18T01:15:55.533323+0000 mgr.y (mgr.14146) 225 : cluster [DBG] pgmap v210: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:59.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:15:59 smithi105 bash[27669]: cluster 2024-02-18T01:15:57.533949+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:15:59 smithi105 bash[19734]: cluster 2024-02-18T01:15:57.533949+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:15:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:15:59 smithi155 bash[20805]: cluster 2024-02-18T01:15:57.533949+0000 mgr.y (mgr.14146) 226 : cluster [DBG] pgmap v211: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:00 smithi105 bash[19734]: audit 2024-02-18T01:16:00.123377+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T01:16:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:00 smithi105 bash[19734]: audit 2024-02-18T01:16:00.125035+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:00 smithi105 bash[27669]: audit 2024-02-18T01:16:00.123377+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T01:16:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:00 smithi105 bash[27669]: audit 2024-02-18T01:16:00.125035+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:00 smithi155 bash[20805]: audit 2024-02-18T01:16:00.123377+0000 mon.a (mon.0) 464 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-02-18T01:16:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:00 smithi155 bash[20805]: audit 2024-02-18T01:16:00.125035+0000 mon.a (mon.0) 465 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:01.449 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:01 smithi155 bash[20805]: cluster 2024-02-18T01:15:59.534582+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:01.449 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:01 smithi155 bash[20805]: cephadm 2024-02-18T01:16:00.126611+0000 mgr.y (mgr.14146) 228 : cephadm [INF] Deploying daemon osd.6 on smithi155 2024-02-18T01:16:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:01 smithi105 bash[19734]: cluster 2024-02-18T01:15:59.534582+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:01 smithi105 bash[19734]: cephadm 2024-02-18T01:16:00.126611+0000 mgr.y (mgr.14146) 228 : cephadm [INF] Deploying daemon osd.6 on smithi155 2024-02-18T01:16:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:01 smithi105 bash[27669]: cluster 2024-02-18T01:15:59.534582+0000 mgr.y (mgr.14146) 227 : cluster [DBG] pgmap v212: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:01 smithi105 bash[27669]: cephadm 2024-02-18T01:16:00.126611+0000 mgr.y (mgr.14146) 228 : cephadm [INF] Deploying daemon osd.6 on smithi155 2024-02-18T01:16:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:02 smithi105 bash[27669]: cluster 2024-02-18T01:16:01.535094+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:02 smithi105 bash[19734]: cluster 2024-02-18T01:16:01.535094+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:02.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:02 smithi155 bash[20805]: cluster 2024-02-18T01:16:01.535094+0000 mgr.y (mgr.14146) 229 : cluster [DBG] pgmap v213: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:05 smithi155 bash[20805]: cluster 2024-02-18T01:16:03.535660+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:05 smithi155 bash[20805]: audit 2024-02-18T01:16:04.058551+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:05 smithi155 bash[20805]: audit 2024-02-18T01:16:04.060871+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[19734]: cluster 2024-02-18T01:16:03.535660+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[19734]: audit 2024-02-18T01:16:04.058551+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[19734]: audit 2024-02-18T01:16:04.060871+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[27669]: cluster 2024-02-18T01:16:03.535660+0000 mgr.y (mgr.14146) 230 : cluster [DBG] pgmap v214: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[27669]: audit 2024-02-18T01:16:04.058551+0000 mon.a (mon.0) 466 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:05 smithi105 bash[27669]: audit 2024-02-18T01:16:04.060871+0000 mon.a (mon.0) 467 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:07.432 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:07 smithi155 bash[20805]: cluster 2024-02-18T01:16:05.536233+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:07 smithi105 bash[19734]: cluster 2024-02-18T01:16:05.536233+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:07 smithi105 bash[27669]: cluster 2024-02-18T01:16:05.536233+0000 mgr.y (mgr.14146) 231 : cluster [DBG] pgmap v215: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:09.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:09 smithi105 bash[27669]: cluster 2024-02-18T01:16:07.536787+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:09 smithi105 bash[19734]: cluster 2024-02-18T01:16:07.536787+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:09 smithi155 bash[20805]: cluster 2024-02-18T01:16:07.536787+0000 mgr.y (mgr.14146) 232 : cluster [DBG] pgmap v216: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: cluster 2024-02-18T01:16:09.537997+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:09.547749+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:09.840111+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:09.842030+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:09.854410+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:10.429508+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:10 smithi155 bash[20805]: audit 2024-02-18T01:16:10.430459+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: cluster 2024-02-18T01:16:09.537997+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:09.547749+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:09.840111+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:09.842030+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:09.854410+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:10.429508+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[19734]: audit 2024-02-18T01:16:10.430459+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: cluster 2024-02-18T01:16:09.537997+0000 mgr.y (mgr.14146) 233 : cluster [DBG] pgmap v217: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:09.547749+0000 mon.a (mon.0) 468 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:09.840111+0000 mon.a (mon.0) 469 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:09.842030+0000 mon.a (mon.0) 470 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:10.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:09.854410+0000 mon.a (mon.0) 471 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:10.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:10.429508+0000 mon.b (mon.1) 13 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:10.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:10 smithi105 bash[27669]: audit 2024-02-18T01:16:10.430459+0000 mon.a (mon.0) 472 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-02-18T01:16:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:12 smithi155 bash[20805]: audit 2024-02-18T01:16:11.077181+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T01:16:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:12 smithi155 bash[20805]: cluster 2024-02-18T01:16:11.077364+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-18T01:16:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:12 smithi155 bash[20805]: audit 2024-02-18T01:16:11.077586+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:12 smithi155 bash[20805]: audit 2024-02-18T01:16:11.078471+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:12 smithi155 bash[20805]: audit 2024-02-18T01:16:11.079248+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[19734]: audit 2024-02-18T01:16:11.077181+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T01:16:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[19734]: cluster 2024-02-18T01:16:11.077364+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-18T01:16:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[19734]: audit 2024-02-18T01:16:11.077586+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[19734]: audit 2024-02-18T01:16:11.078471+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[19734]: audit 2024-02-18T01:16:11.079248+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[27669]: audit 2024-02-18T01:16:11.077181+0000 mon.a (mon.0) 473 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-02-18T01:16:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[27669]: cluster 2024-02-18T01:16:11.077364+0000 mon.a (mon.0) 474 : cluster [DBG] osdmap e41: 7 total, 6 up, 7 in 2024-02-18T01:16:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[27669]: audit 2024-02-18T01:16:11.077586+0000 mon.a (mon.0) 475 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[27669]: audit 2024-02-18T01:16:11.078471+0000 mon.b (mon.1) 14 : audit [INF] from='osd.6 v1:172.21.15.155:6808/98819544' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:12 smithi105 bash[27669]: audit 2024-02-18T01:16:11.079248+0000 mon.a (mon.0) 476 : audit [INF] from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:12.557 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.553+0000 7fa85bfff700 1 -- 172.21.15.155:0/1899383558 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 1961374907) 0x7fa860000b90 con 0x7fa85c0621a0 2024-02-18T01:16:12.558 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 6 on host 'smithi155' 2024-02-18T01:16:12.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 >> v1:172.21.15.105:6800/3087430119 conn(0x7fa85c0621a0 legacy=0x7fa85c064660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:12.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 >> v1:172.21.15.155:6789/0 conn(0x7fa8740fe820 legacy=0x7fa874111550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:12.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 shutdown_connections 2024-02-18T01:16:12.561 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 >> 172.21.15.155:0/1899383558 conn(0x7fa8740f1e60 msgr2=0x7fa8740fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:16:12.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 shutdown_connections 2024-02-18T01:16:12.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:12.557+0000 7fa879759700 1 -- 172.21.15.155:0/1899383558 wait complete. 2024-02-18T01:16:13.153 DEBUG:teuthology.orchestra.run.smithi155:osd.6> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.6.service 2024-02-18T01:16:13.155 INFO:tasks.cephadm:Deploying osd.7 on smithi155 with /dev/vg_nvme/lv_1... 2024-02-18T01:16:13.156 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-02-18T01:16:13.168 INFO:journalctl@ceph.osd.6.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:16:13.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: cluster 2024-02-18T01:16:11.538541+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: audit 2024-02-18T01:16:12.080653+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: cluster 2024-02-18T01:16:12.080769+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: audit 2024-02-18T01:16:12.081469+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: audit 2024-02-18T01:16:12.087636+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: audit 2024-02-18T01:16:12.555526+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[19734]: audit 2024-02-18T01:16:12.582937+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: cluster 2024-02-18T01:16:11.538541+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: audit 2024-02-18T01:16:12.080653+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: cluster 2024-02-18T01:16:12.080769+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T01:16:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: audit 2024-02-18T01:16:12.081469+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: audit 2024-02-18T01:16:12.087636+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: audit 2024-02-18T01:16:12.555526+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:13 smithi105 bash[27669]: audit 2024-02-18T01:16:12.582937+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: cluster 2024-02-18T01:16:11.538541+0000 mgr.y (mgr.14146) 234 : cluster [DBG] pgmap v219: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: audit 2024-02-18T01:16:12.080653+0000 mon.a (mon.0) 477 : audit [INF] from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: cluster 2024-02-18T01:16:12.080769+0000 mon.a (mon.0) 478 : cluster [DBG] osdmap e42: 7 total, 6 up, 7 in 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: audit 2024-02-18T01:16:12.081469+0000 mon.a (mon.0) 479 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: audit 2024-02-18T01:16:12.087636+0000 mon.a (mon.0) 480 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: audit 2024-02-18T01:16:12.555526+0000 mon.a (mon.0) 481 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:13.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:13 smithi155 bash[20805]: audit 2024-02-18T01:16:12.582937+0000 mon.a (mon.0) 482 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:14.454 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: cluster 2024-02-18T01:16:11.429893+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:14.454 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: cluster 2024-02-18T01:16:11.430010+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:14.454 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: cluster 2024-02-18T01:16:13.097122+0000 mon.a (mon.0) 483 : cluster [INF] osd.6 v1:172.21.15.155:6808/98819544 boot 2024-02-18T01:16:14.454 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: cluster 2024-02-18T01:16:13.097278+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-18T01:16:14.455 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: audit 2024-02-18T01:16:13.097979+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:14.455 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:14 smithi155 bash[20805]: cluster 2024-02-18T01:16:14.088023+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-18T01:16:14.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: cluster 2024-02-18T01:16:11.429893+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:14.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: cluster 2024-02-18T01:16:11.430010+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: cluster 2024-02-18T01:16:13.097122+0000 mon.a (mon.0) 483 : cluster [INF] osd.6 v1:172.21.15.155:6808/98819544 boot 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: cluster 2024-02-18T01:16:13.097278+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: audit 2024-02-18T01:16:13.097979+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[19734]: cluster 2024-02-18T01:16:14.088023+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: cluster 2024-02-18T01:16:11.429893+0000 osd.6 (osd.6) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: cluster 2024-02-18T01:16:11.430010+0000 osd.6 (osd.6) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: cluster 2024-02-18T01:16:13.097122+0000 mon.a (mon.0) 483 : cluster [INF] osd.6 v1:172.21.15.155:6808/98819544 boot 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: cluster 2024-02-18T01:16:13.097278+0000 mon.a (mon.0) 484 : cluster [DBG] osdmap e43: 7 total, 7 up, 7 in 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: audit 2024-02-18T01:16:13.097979+0000 mon.a (mon.0) 485 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:16:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:14 smithi105 bash[27669]: cluster 2024-02-18T01:16:14.088023+0000 mon.a (mon.0) 486 : cluster [DBG] osdmap e44: 7 total, 7 up, 7 in 2024-02-18T01:16:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:15 smithi105 bash[19734]: cluster 2024-02-18T01:16:13.539154+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:15.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:15 smithi105 bash[19734]: cluster 2024-02-18T01:16:15.090259+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-18T01:16:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:15 smithi105 bash[27669]: cluster 2024-02-18T01:16:13.539154+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:15 smithi105 bash[27669]: cluster 2024-02-18T01:16:15.090259+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-18T01:16:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:15 smithi155 bash[20805]: cluster 2024-02-18T01:16:13.539154+0000 mgr.y (mgr.14146) 235 : cluster [DBG] pgmap v222: 1 pgs: 1 active+clean; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-02-18T01:16:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:15 smithi155 bash[20805]: cluster 2024-02-18T01:16:15.090259+0000 mon.a (mon.0) 487 : cluster [DBG] osdmap e45: 7 total, 7 up, 7 in 2024-02-18T01:16:16.472 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-02-18T01:16:16.472 INFO:teuthology.orchestra.run.smithi155.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-02-18T01:16:16.472 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10+0 records in 2024-02-18T01:16:16.472 INFO:teuthology.orchestra.run.smithi155.stderr:10+0 records out 2024-02-18T01:16:16.473 INFO:teuthology.orchestra.run.smithi155.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0174549 s, 601 MB/s 2024-02-18T01:16:16.473 INFO:teuthology.orchestra.run.smithi155.stderr:--> Zapping successful for: 2024-02-18T01:16:17.113 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch daemon add osd smithi155:vg_nvme/lv_1 2024-02-18T01:16:17.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:17 smithi105 bash[27669]: cluster 2024-02-18T01:16:15.539703+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:17.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:17 smithi105 bash[19734]: cluster 2024-02-18T01:16:15.539703+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:17.501 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:17 smithi155 bash[20805]: cluster 2024-02-18T01:16:15.539703+0000 mgr.y (mgr.14146) 236 : cluster [DBG] pgmap v225: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:19 smithi155 bash[20805]: cluster 2024-02-18T01:16:17.540320+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:19 smithi105 bash[19734]: cluster 2024-02-18T01:16:17.540320+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:19 smithi105 bash[27669]: cluster 2024-02-18T01:16:17.540320+0000 mgr.y (mgr.14146) 237 : cluster [DBG] pgmap v226: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:21.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/2883429801 >> v1:172.21.15.155:6789/0 conn(0x7f39140f5e90 legacy=0x7f39140f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:21.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/2883429801 shutdown_connections 2024-02-18T01:16:21.283 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/2883429801 >> 172.21.15.155:0/2883429801 conn(0x7f39140f1610 msgr2=0x7f39140f3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:16:21.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/2883429801 shutdown_connections 2024-02-18T01:16:21.284 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/2883429801 wait complete. 2024-02-18T01:16:21.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 Processor -- start 2024-02-18T01:16:21.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- start start 2024-02-18T01:16:21.288 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f39141063f0 con 0x7f39140f5e90 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3914111bb0 con 0x7f39140fa350 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3914112d90 con 0x7f39140fdfb0 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3912ffd700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f39140fa350 0x7f391410bd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:52156/0 (socket says 172.21.15.155:52156) 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3912ffd700 1 -- 172.21.15.155:0/3793579185 learned_addr learned my addr 172.21.15.155:0/3793579185 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2752126283 0 0) 0x7f3914111bb0 con 0x7f39140fa350 2024-02-18T01:16:21.289 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38fc003030 con 0x7f39140fa350 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 849788886 0 0) 0x7f39141063f0 con 0x7f39140f5e90 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3914111bb0 con 0x7f39140f5e90 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952485276 0 0) 0x7f3914112d90 con 0x7f39140fdfb0 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39141063f0 con 0x7f39140fdfb0 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 141912911 0 0) 0x7f38fc003030 con 0x7f39140fa350 2024-02-18T01:16:21.290 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3914112d90 con 0x7f39140fa350 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f3908001980 con 0x7f39140fa350 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1923202114 0 0) 0x7f3914111bb0 con 0x7f39140f5e90 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f38fc003030 con 0x7f39140f5e90 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f3904002ec0 con 0x7f39140f5e90 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2595035090 0 0) 0x7f39141063f0 con 0x7f39140fdfb0 2024-02-18T01:16:21.291 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3914111bb0 con 0x7f39140fdfb0 2024-02-18T01:16:21.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f390c001a30 con 0x7f39140fdfb0 2024-02-18T01:16:21.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1137604342 0 0) 0x7f3914112d90 con 0x7f39140fa350 2024-02-18T01:16:21.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 >> v1:172.21.15.105:6790/0 conn(0x7f39140fdfb0 legacy=0x7f391410f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:21.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 >> v1:172.21.15.105:6789/0 conn(0x7f39140f5e90 legacy=0x7f3914105800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:21.292 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914113f70 con 0x7f39140fa350 2024-02-18T01:16:21.295 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3914111da0 con 0x7f39140fa350 2024-02-18T01:16:21.295 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.280+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3914112290 con 0x7f39140fa350 2024-02-18T01:16:21.295 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.284+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f3908001c40 con 0x7f39140fa350 2024-02-18T01:16:21.295 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.284+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f3908005b80 con 0x7f39140fa350 2024-02-18T01:16:21.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.284+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3900004f20 con 0x7f39140fa350 2024-02-18T01:16:21.296 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.284+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f3908005d40 con 0x7f39140fa350 2024-02-18T01:16:21.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.284+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(45..45 src has 1..45) v4 ==== 3762+0+0 (unknown 1322287908 0 0) 0x7f3908077b60 con 0x7f39140fa350 2024-02-18T01:16:21.297 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.292+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f3908048560 con 0x7f39140fa350 2024-02-18T01:16:21.488 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:21.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3900000b90 con 0x7f38fc064440 2024-02-18T01:16:21.551 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:21 smithi105 bash[19734]: cluster 2024-02-18T01:16:19.540865+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:21.552 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:21 smithi155 bash[20805]: cluster 2024-02-18T01:16:19.540865+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:21.552 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:21 smithi105 bash[27669]: cluster 2024-02-18T01:16:19.540865+0000 mgr.y (mgr.14146) 238 : cluster [DBG] pgmap v227: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:22 smithi155 bash[20805]: audit 2024-02-18T01:16:21.496022+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:16:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:22 smithi155 bash[20805]: audit 2024-02-18T01:16:21.502471+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:16:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:22 smithi155 bash[20805]: audit 2024-02-18T01:16:21.504294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[19734]: audit 2024-02-18T01:16:21.496022+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[19734]: audit 2024-02-18T01:16:21.502471+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[19734]: audit 2024-02-18T01:16:21.504294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[27669]: audit 2024-02-18T01:16:21.496022+0000 mon.a (mon.0) 488 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[27669]: audit 2024-02-18T01:16:21.502471+0000 mon.a (mon.0) 489 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-02-18T01:16:22.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:22 smithi105 bash[27669]: audit 2024-02-18T01:16:21.504294+0000 mon.a (mon.0) 490 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:21.488940+0000 mgr.y (mgr.14146) 239 : audit [DBG] from='client.24310 v1:172.21.15.155:0/3793579185' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: cluster 2024-02-18T01:16:21.541465+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.811485+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.813602+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.815039+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.816490+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.825269+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.827042+0000 mon.a (mon.0) 496 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.828464+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[19734]: audit 2024-02-18T01:16:22.836919+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:21.488940+0000 mgr.y (mgr.14146) 239 : audit [DBG] from='client.24310 v1:172.21.15.155:0/3793579185' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: cluster 2024-02-18T01:16:21.541465+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.811485+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.813602+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.815039+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.816490+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.825269+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.827042+0000 mon.a (mon.0) 496 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.828464+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:23 smithi105 bash[27669]: audit 2024-02-18T01:16:22.836919+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:21.488940+0000 mgr.y (mgr.14146) 239 : audit [DBG] from='client.24310 v1:172.21.15.155:0/3793579185' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi155:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: cluster 2024-02-18T01:16:21.541465+0000 mgr.y (mgr.14146) 240 : cluster [DBG] pgmap v228: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.811485+0000 mon.a (mon.0) 491 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.813602+0000 mon.a (mon.0) 492 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.815039+0000 mon.a (mon.0) 493 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.816490+0000 mon.a (mon.0) 494 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.825269+0000 mon.a (mon.0) 495 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.827042+0000 mon.a (mon.0) 496 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.828464+0000 mon.a (mon.0) 497 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:23.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:23 smithi155 bash[20805]: audit 2024-02-18T01:16:22.836919+0000 mon.a (mon.0) 498 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:24 smithi105 bash[19734]: cephadm 2024-02-18T01:16:22.801786+0000 mgr.y (mgr.14146) 241 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:16:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:24 smithi105 bash[19734]: cephadm 2024-02-18T01:16:22.817515+0000 mgr.y (mgr.14146) 242 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 5772M 2024-02-18T01:16:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:24 smithi105 bash[27669]: cephadm 2024-02-18T01:16:22.801786+0000 mgr.y (mgr.14146) 241 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:16:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:24 smithi105 bash[27669]: cephadm 2024-02-18T01:16:22.817515+0000 mgr.y (mgr.14146) 242 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 5772M 2024-02-18T01:16:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:24 smithi155 bash[20805]: cephadm 2024-02-18T01:16:22.801786+0000 mgr.y (mgr.14146) 241 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:16:24.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:24 smithi155 bash[20805]: cephadm 2024-02-18T01:16:22.817515+0000 mgr.y (mgr.14146) 242 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 5772M 2024-02-18T01:16:25.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:25 smithi105 bash[19734]: cluster 2024-02-18T01:16:23.541998+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:25.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:25 smithi105 bash[27669]: cluster 2024-02-18T01:16:23.541998+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:25.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:25 smithi155 bash[20805]: cluster 2024-02-18T01:16:23.541998+0000 mgr.y (mgr.14146) 243 : cluster [DBG] pgmap v229: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:27.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:27 smithi105 bash[19734]: cluster 2024-02-18T01:16:25.542586+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:27 smithi105 bash[27669]: cluster 2024-02-18T01:16:25.542586+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:27 smithi155 bash[20805]: cluster 2024-02-18T01:16:25.542586+0000 mgr.y (mgr.14146) 244 : cluster [DBG] pgmap v230: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:28.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:28 smithi105 bash[19734]: cluster 2024-02-18T01:16:27.543238+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:28.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:28 smithi105 bash[27669]: cluster 2024-02-18T01:16:27.543238+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:28 smithi155 bash[20805]: cluster 2024-02-18T01:16:27.543238+0000 mgr.y (mgr.14146) 245 : cluster [DBG] pgmap v231: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: cluster 2024-02-18T01:16:29.543888+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: audit 2024-02-18T01:16:29.812608+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.155:0/3699095272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: audit 2024-02-18T01:16:29.813497+0000 mon.a (mon.0) 499 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: audit 2024-02-18T01:16:29.822550+0000 mon.a (mon.0) 500 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]': finished 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: cluster 2024-02-18T01:16:29.822625+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: audit 2024-02-18T01:16:29.822886+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[19734]: audit 2024-02-18T01:16:30.418696+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.155:0/4244744718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: cluster 2024-02-18T01:16:29.543888+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: audit 2024-02-18T01:16:29.812608+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.155:0/3699095272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: audit 2024-02-18T01:16:29.813497+0000 mon.a (mon.0) 499 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: audit 2024-02-18T01:16:29.822550+0000 mon.a (mon.0) 500 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]': finished 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: cluster 2024-02-18T01:16:29.822625+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: audit 2024-02-18T01:16:29.822886+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:30.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:30 smithi105 bash[27669]: audit 2024-02-18T01:16:30.418696+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.155:0/4244744718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: cluster 2024-02-18T01:16:29.543888+0000 mgr.y (mgr.14146) 246 : cluster [DBG] pgmap v232: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: audit 2024-02-18T01:16:29.812608+0000 mon.b (mon.1) 15 : audit [INF] from='client.? v1:172.21.15.155:0/3699095272' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: audit 2024-02-18T01:16:29.813497+0000 mon.a (mon.0) 499 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]: dispatch 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: audit 2024-02-18T01:16:29.822550+0000 mon.a (mon.0) 500 : audit [INF] from='client.24316 ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0b9beb5b-519d-4ef0-9768-cf509db5d07d"}]': finished 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: cluster 2024-02-18T01:16:29.822625+0000 mon.a (mon.0) 501 : cluster [DBG] osdmap e46: 8 total, 7 up, 8 in 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: audit 2024-02-18T01:16:29.822886+0000 mon.a (mon.0) 502 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:30 smithi155 bash[20805]: audit 2024-02-18T01:16:30.418696+0000 mon.b (mon.1) 16 : audit [DBG] from='client.? v1:172.21.15.155:0/4244744718' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-02-18T01:16:32.873 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:32 smithi155 bash[20805]: cluster 2024-02-18T01:16:31.544481+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:32 smithi105 bash[19734]: cluster 2024-02-18T01:16:31.544481+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:32 smithi105 bash[27669]: cluster 2024-02-18T01:16:31.544481+0000 mgr.y (mgr.14146) 247 : cluster [DBG] pgmap v234: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:34 smithi105 bash[19734]: cluster 2024-02-18T01:16:33.545093+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:34.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:34 smithi105 bash[27669]: cluster 2024-02-18T01:16:33.545093+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:34.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:34 smithi155 bash[20805]: cluster 2024-02-18T01:16:33.545093+0000 mgr.y (mgr.14146) 248 : cluster [DBG] pgmap v235: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:36 smithi105 bash[19734]: cluster 2024-02-18T01:16:35.545685+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:36 smithi105 bash[27669]: cluster 2024-02-18T01:16:35.545685+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:36 smithi155 bash[20805]: cluster 2024-02-18T01:16:35.545685+0000 mgr.y (mgr.14146) 249 : cluster [DBG] pgmap v236: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:38 smithi105 bash[19734]: cluster 2024-02-18T01:16:37.546347+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:38 smithi105 bash[27669]: cluster 2024-02-18T01:16:37.546347+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:38 smithi155 bash[20805]: cluster 2024-02-18T01:16:37.546347+0000 mgr.y (mgr.14146) 250 : cluster [DBG] pgmap v237: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:40.907 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:40 smithi155 bash[20805]: cluster 2024-02-18T01:16:39.547067+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:40 smithi105 bash[19734]: cluster 2024-02-18T01:16:39.547067+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:40 smithi105 bash[27669]: cluster 2024-02-18T01:16:39.547067+0000 mgr.y (mgr.14146) 251 : cluster [DBG] pgmap v238: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[19734]: cluster 2024-02-18T01:16:41.547690+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[19734]: audit 2024-02-18T01:16:41.958878+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[19734]: audit 2024-02-18T01:16:41.960478+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[19734]: cephadm 2024-02-18T01:16:41.961917+0000 mgr.y (mgr.14146) 253 : cephadm [INF] Deploying daemon osd.7 on smithi155 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[27669]: cluster 2024-02-18T01:16:41.547690+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[27669]: audit 2024-02-18T01:16:41.958878+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[27669]: audit 2024-02-18T01:16:41.960478+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:42 smithi105 bash[27669]: cephadm 2024-02-18T01:16:41.961917+0000 mgr.y (mgr.14146) 253 : cephadm [INF] Deploying daemon osd.7 on smithi155 2024-02-18T01:16:42.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:42 smithi155 bash[20805]: cluster 2024-02-18T01:16:41.547690+0000 mgr.y (mgr.14146) 252 : cluster [DBG] pgmap v239: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:42.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:42 smithi155 bash[20805]: audit 2024-02-18T01:16:41.958878+0000 mon.a (mon.0) 503 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-02-18T01:16:42.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:42 smithi155 bash[20805]: audit 2024-02-18T01:16:41.960478+0000 mon.a (mon.0) 504 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:42.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:42 smithi155 bash[20805]: cephadm 2024-02-18T01:16:41.961917+0000 mgr.y (mgr.14146) 253 : cephadm [INF] Deploying daemon osd.7 on smithi155 2024-02-18T01:16:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:44 smithi105 bash[19734]: cluster 2024-02-18T01:16:43.548248+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:44 smithi105 bash[27669]: cluster 2024-02-18T01:16:43.548248+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:44 smithi155 bash[20805]: cluster 2024-02-18T01:16:43.548248+0000 mgr.y (mgr.14146) 254 : cluster [DBG] pgmap v240: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:46 smithi155 bash[20805]: cluster 2024-02-18T01:16:45.548821+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:46.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:46 smithi155 bash[20805]: audit 2024-02-18T01:16:45.870139+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:46.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:46 smithi155 bash[20805]: audit 2024-02-18T01:16:45.878780+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:46.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[27669]: cluster 2024-02-18T01:16:45.548821+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[27669]: audit 2024-02-18T01:16:45.870139+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[27669]: audit 2024-02-18T01:16:45.878780+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[19734]: cluster 2024-02-18T01:16:45.548821+0000 mgr.y (mgr.14146) 255 : cluster [DBG] pgmap v241: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[19734]: audit 2024-02-18T01:16:45.870139+0000 mon.a (mon.0) 505 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:46 smithi105 bash[19734]: audit 2024-02-18T01:16:45.878780+0000 mon.a (mon.0) 506 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:48.955 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:48 smithi155 bash[20805]: cluster 2024-02-18T01:16:47.549432+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:48 smithi105 bash[19734]: cluster 2024-02-18T01:16:47.549432+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:48 smithi105 bash[27669]: cluster 2024-02-18T01:16:47.549432+0000 mgr.y (mgr.14146) 256 : cluster [DBG] pgmap v242: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:50 smithi105 bash[19734]: cluster 2024-02-18T01:16:49.550020+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:50.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:50 smithi105 bash[27669]: cluster 2024-02-18T01:16:49.550020+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:50.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:50 smithi155 bash[20805]: cluster 2024-02-18T01:16:49.550020+0000 mgr.y (mgr.14146) 257 : cluster [DBG] pgmap v243: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[19734]: audit 2024-02-18T01:16:51.336930+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[19734]: audit 2024-02-18T01:16:51.338632+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[19734]: audit 2024-02-18T01:16:51.340177+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[19734]: audit 2024-02-18T01:16:51.349400+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[19734]: cluster 2024-02-18T01:16:51.550572+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[27669]: audit 2024-02-18T01:16:51.336930+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[27669]: audit 2024-02-18T01:16:51.338632+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[27669]: audit 2024-02-18T01:16:51.340177+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[27669]: audit 2024-02-18T01:16:51.349400+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:52 smithi105 bash[27669]: cluster 2024-02-18T01:16:51.550572+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:52 smithi155 bash[20805]: audit 2024-02-18T01:16:51.336930+0000 mon.a (mon.0) 507 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:52 smithi155 bash[20805]: audit 2024-02-18T01:16:51.338632+0000 mon.a (mon.0) 508 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:16:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:52 smithi155 bash[20805]: audit 2024-02-18T01:16:51.340177+0000 mon.a (mon.0) 509 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:16:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:52 smithi155 bash[20805]: audit 2024-02-18T01:16:51.349400+0000 mon.a (mon.0) 510 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:52 smithi155 bash[20805]: cluster 2024-02-18T01:16:51.550572+0000 mgr.y (mgr.14146) 258 : cluster [DBG] pgmap v244: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:53 smithi105 bash[19734]: audit 2024-02-18T01:16:52.563419+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:53 smithi105 bash[19734]: audit 2024-02-18T01:16:52.564093+0000 mon.a (mon.0) 511 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:53 smithi105 bash[27669]: audit 2024-02-18T01:16:52.563419+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:53 smithi105 bash[27669]: audit 2024-02-18T01:16:52.564093+0000 mon.a (mon.0) 511 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:53.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:53 smithi155 bash[20805]: audit 2024-02-18T01:16:52.563419+0000 mon.b (mon.1) 17 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:53 smithi155 bash[20805]: audit 2024-02-18T01:16:52.564093+0000 mon.a (mon.0) 511 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-02-18T01:16:54.485 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.480+0000 7f3910ff9700 1 -- 172.21.15.155:0/3793579185 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (unknown 0 0 687725071) 0x7f3900000b90 con 0x7f38fc064440 2024-02-18T01:16:54.486 INFO:teuthology.orchestra.run.smithi155.stdout:Created osd(s) 7 on host 'smithi155' 2024-02-18T01:16:54.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 >> v1:172.21.15.105:6800/3087430119 conn(0x7f38fc064440 legacy=0x7f38fc066900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:54.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 >> v1:172.21.15.155:6789/0 conn(0x7f39140fa350 legacy=0x7f391410bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:54.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 shutdown_connections 2024-02-18T01:16:54.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 >> 172.21.15.155:0/3793579185 conn(0x7f39140f1610 msgr2=0x7f39141037d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:16:54.489 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 shutdown_connections 2024-02-18T01:16:54.490 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:16:54.484+0000 7f391a1e8700 1 -- 172.21.15.155:0/3793579185 wait complete. 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: audit 2024-02-18T01:16:53.361327+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: cluster 2024-02-18T01:16:53.361581+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: audit 2024-02-18T01:16:53.361774+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: audit 2024-02-18T01:16:53.362425+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: audit 2024-02-18T01:16:53.362973+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[19734]: cluster 2024-02-18T01:16:53.551124+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:54.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: audit 2024-02-18T01:16:53.361327+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T01:16:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: cluster 2024-02-18T01:16:53.361581+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-18T01:16:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: audit 2024-02-18T01:16:53.361774+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: audit 2024-02-18T01:16:53.362425+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: audit 2024-02-18T01:16:53.362973+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:54 smithi105 bash[27669]: cluster 2024-02-18T01:16:53.551124+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:54.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: audit 2024-02-18T01:16:53.361327+0000 mon.a (mon.0) 512 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-02-18T01:16:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: cluster 2024-02-18T01:16:53.361581+0000 mon.a (mon.0) 513 : cluster [DBG] osdmap e47: 8 total, 7 up, 8 in 2024-02-18T01:16:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: audit 2024-02-18T01:16:53.361774+0000 mon.a (mon.0) 514 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: audit 2024-02-18T01:16:53.362425+0000 mon.b (mon.1) 18 : audit [INF] from='osd.7 v1:172.21.15.155:6812/1643073216' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: audit 2024-02-18T01:16:53.362973+0000 mon.a (mon.0) 515 : audit [INF] from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]: dispatch 2024-02-18T01:16:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:54 smithi155 bash[20805]: cluster 2024-02-18T01:16:53.551124+0000 mgr.y (mgr.14146) 259 : cluster [DBG] pgmap v246: 1 pgs: 1 active+clean; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-02-18T01:16:55.222 DEBUG:teuthology.orchestra.run.smithi155:osd.7> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.7.service 2024-02-18T01:16:55.225 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-02-18T01:16:55.225 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd stat -f json 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:54.365028+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: cluster 2024-02-18T01:16:54.365501+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:54.366274+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:54.373634+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:54.476785+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:54.482911+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: cluster 2024-02-18T01:16:55.365914+0000 mon.a (mon.0) 522 : cluster [INF] osd.7 v1:172.21.15.155:6812/1643073216 boot 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: cluster 2024-02-18T01:16:55.365949+0000 mon.a (mon.0) 523 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-18T01:16:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[19734]: audit 2024-02-18T01:16:55.366140+0000 mon.a (mon.0) 524 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:54.365028+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: cluster 2024-02-18T01:16:54.365501+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:54.366274+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:54.373634+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:54.476785+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:54.482911+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: cluster 2024-02-18T01:16:55.365914+0000 mon.a (mon.0) 522 : cluster [INF] osd.7 v1:172.21.15.155:6812/1643073216 boot 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: cluster 2024-02-18T01:16:55.365949+0000 mon.a (mon.0) 523 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-18T01:16:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:55 smithi105 bash[27669]: audit 2024-02-18T01:16:55.366140+0000 mon.a (mon.0) 524 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:54.365028+0000 mon.a (mon.0) 516 : audit [INF] from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi155", "root=default"]}]': finished 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: cluster 2024-02-18T01:16:54.365501+0000 mon.a (mon.0) 517 : cluster [DBG] osdmap e48: 8 total, 7 up, 8 in 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:54.366274+0000 mon.a (mon.0) 518 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:54.373634+0000 mon.a (mon.0) 519 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:54.476785+0000 mon.a (mon.0) 520 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:54.482911+0000 mon.a (mon.0) 521 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: cluster 2024-02-18T01:16:55.365914+0000 mon.a (mon.0) 522 : cluster [INF] osd.7 v1:172.21.15.155:6812/1643073216 boot 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: cluster 2024-02-18T01:16:55.365949+0000 mon.a (mon.0) 523 : cluster [DBG] osdmap e49: 8 total, 8 up, 8 in 2024-02-18T01:16:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:55 smithi155 bash[20805]: audit 2024-02-18T01:16:55.366140+0000 mon.a (mon.0) 524 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:16:55.495 INFO:journalctl@ceph.osd.7.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:16:56.676 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[19734]: cluster 2024-02-18T01:16:53.541457+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[19734]: cluster 2024-02-18T01:16:53.541608+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[19734]: cluster 2024-02-18T01:16:55.551716+0000 mgr.y (mgr.14146) 260 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[19734]: cluster 2024-02-18T01:16:56.370747+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[27669]: cluster 2024-02-18T01:16:53.541457+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[27669]: cluster 2024-02-18T01:16:53.541608+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[27669]: cluster 2024-02-18T01:16:55.551716+0000 mgr.y (mgr.14146) 260 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:56.677 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:56 smithi105 bash[27669]: cluster 2024-02-18T01:16:56.370747+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-18T01:16:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:56 smithi155 bash[20805]: cluster 2024-02-18T01:16:53.541457+0000 osd.7 (osd.7) 1 : cluster [DBG] purged_snaps scrub starts 2024-02-18T01:16:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:56 smithi155 bash[20805]: cluster 2024-02-18T01:16:53.541608+0000 osd.7 (osd.7) 2 : cluster [DBG] purged_snaps scrub ok 2024-02-18T01:16:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:56 smithi155 bash[20805]: cluster 2024-02-18T01:16:55.551716+0000 mgr.y (mgr.14146) 260 : cluster [DBG] pgmap v249: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:56 smithi155 bash[20805]: cluster 2024-02-18T01:16:56.370747+0000 mon.a (mon.0) 525 : cluster [DBG] osdmap e50: 8 total, 8 up, 8 in 2024-02-18T01:16:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3352289290 >> v1:172.21.15.105:6789/0 conn(0x7feeb00f7d00 legacy=0x7feeb00f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3352289290 shutdown_connections 2024-02-18T01:16:58.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3352289290 >> 172.21.15.105:0/3352289290 conn(0x7feeb00f1630 msgr2=0x7feeb00f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3352289290 shutdown_connections 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3352289290 wait complete. 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 Processor -- start 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- start start 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feeb010a400 con 0x7feeb00f6940 2024-02-18T01:16:58.331 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feeb0115c60 con 0x7feeb00f74e0 2024-02-18T01:16:58.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7feeb0116e40 con 0x7feeb00f7d00 2024-02-18T01:16:58.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeaeffd700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7feeb00f6940 0x7feeb0109830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:60504/0 (socket says 172.21.15.105:60504) 2024-02-18T01:16:58.332 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeaeffd700 1 -- 172.21.15.105:0/3329026004 learned_addr learned my addr 172.21.15.105:0/3329026004 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919992658 0 0) 0x7feeb0116e40 con 0x7feeb00f7d00 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c003030 con 0x7feeb00f7d00 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2998218250 0 0) 0x7feeb0115c60 con 0x7feeb00f74e0 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeb0116e40 con 0x7feeb00f74e0 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3416807361 0 0) 0x7fee9c003030 con 0x7feeb00f7d00 2024-02-18T01:16:58.333 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feeb0115c60 con 0x7feeb00f7d00 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7feea00018c0 con 0x7feeb00f7d00 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845870596 0 0) 0x7feeb010a400 con 0x7feeb00f6940 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee9c003030 con 0x7feeb00f6940 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3398686847 0 0) 0x7feeb0116e40 con 0x7feeb00f74e0 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7feeb010a400 con 0x7feeb00f74e0 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fee980016a0 con 0x7feeb00f74e0 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1692017690 0 0) 0x7feeb0115c60 con 0x7feeb00f7d00 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 >> v1:172.21.15.155:6789/0 conn(0x7feeb00f74e0 legacy=0x7feeb010fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 >> v1:172.21.15.105:6789/0 conn(0x7feeb00f6940 legacy=0x7feeb0109830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeb0118020 con 0x7feeb00f7d00 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb0115e50 con 0x7feeb00f7d00 2024-02-18T01:16:58.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.328+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb0116340 con 0x7feeb00f7d00 2024-02-18T01:16:58.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.332+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7feea0001f80 con 0x7feeb00f7d00 2024-02-18T01:16:58.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.332+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7feea0003260 con 0x7feeb00f7d00 2024-02-18T01:16:58.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.332+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee90004f20 con 0x7feeb00f7d00 2024-02-18T01:16:58.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.332+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7feea0016130 con 0x7feeb00f7d00 2024-02-18T01:16:58.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.332+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4054+0+0 (unknown 32312736 0 0) 0x7feea0075360 con 0x7feeb00f7d00 2024-02-18T01:16:58.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.340+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7feea0045c40 con 0x7feeb00f7d00 2024-02-18T01:16:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.528+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fee900051d0 con 0x7feeb00f7d00 2024-02-18T01:16:58.531 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.528+0000 7feea7fff700 1 -- 172.21.15.105:0/3329026004 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v51) v1 ==== 74+0+130 (unknown 4287931526 0 1815710671) 0x7feea0047b70 con 0x7feeb00f7d00 2024-02-18T01:16:58.532 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:16:58.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 >> v1:172.21.15.105:6800/3087430119 conn(0x7fee9c0621b0 legacy=0x7fee9c064670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:58.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 >> v1:172.21.15.105:6790/0 conn(0x7feeb00f7d00 legacy=0x7feeb0113530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:16:58.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 shutdown_connections 2024-02-18T01:16:58.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 >> 172.21.15.105:0/3329026004 conn(0x7feeb00f1630 msgr2=0x7feeb00f3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:16:58.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 shutdown_connections 2024-02-18T01:16:58.536 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:16:58.532+0000 7feeb589a700 1 -- 172.21.15.105:0/3329026004 wait complete. 2024-02-18T01:16:58.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:58 smithi105 bash[19734]: cluster 2024-02-18T01:16:57.386293+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-18T01:16:58.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:58 smithi105 bash[19734]: cluster 2024-02-18T01:16:57.552378+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v252: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:58 smithi105 bash[27669]: cluster 2024-02-18T01:16:57.386293+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-18T01:16:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:58 smithi105 bash[27669]: cluster 2024-02-18T01:16:57.552378+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v252: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:58 smithi155 bash[20805]: cluster 2024-02-18T01:16:57.386293+0000 mon.a (mon.0) 526 : cluster [DBG] osdmap e51: 8 total, 8 up, 8 in 2024-02-18T01:16:58.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:58 smithi155 bash[20805]: cluster 2024-02-18T01:16:57.552378+0000 mgr.y (mgr.14146) 261 : cluster [DBG] pgmap v252: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:16:59.173 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":51,"num_osds":8,"num_up_osds":8,"osd_up_since":1708219015,"num_in_osds":8,"osd_in_since":1708218989,"num_remapped_pgs":0} 2024-02-18T01:16:59.174 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi105 2024-02-18T01:16:59.174 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi105=foo.a' 2024-02-18T01:16:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:59 smithi105 bash[19734]: cluster 2024-02-18T01:16:58.383147+0000 mon.a (mon.0) 527 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-18T01:16:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:16:59 smithi105 bash[19734]: audit 2024-02-18T01:16:58.530835+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? v1:172.21.15.105:0/3329026004' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T01:16:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:59 smithi105 bash[27669]: cluster 2024-02-18T01:16:58.383147+0000 mon.a (mon.0) 527 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-18T01:16:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:16:59 smithi105 bash[27669]: audit 2024-02-18T01:16:58.530835+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? v1:172.21.15.105:0/3329026004' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T01:16:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:59 smithi155 bash[20805]: cluster 2024-02-18T01:16:58.383147+0000 mon.a (mon.0) 527 : cluster [WRN] Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-02-18T01:16:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:16:59 smithi155 bash[20805]: audit 2024-02-18T01:16:58.530835+0000 mon.c (mon.2) 17 : audit [DBG] from='client.? v1:172.21.15.105:0/3329026004' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-02-18T01:17:00.700 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:00 smithi155 bash[20805]: cluster 2024-02-18T01:16:59.553032+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v253: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:00.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:00 smithi105 bash[19734]: cluster 2024-02-18T01:16:59.553032+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v253: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:00.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:00 smithi105 bash[27669]: cluster 2024-02-18T01:16:59.553032+0000 mgr.y (mgr.14146) 262 : cluster [DBG] pgmap v253: 1 pgs: 1 remapped+peering; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:01 smithi105 bash[19734]: cluster 2024-02-18T01:17:01.604857+0000 mon.a (mon.0) 528 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-18T01:17:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:01 smithi105 bash[19734]: cluster 2024-02-18T01:17:01.604942+0000 mon.a (mon.0) 529 : cluster [INF] Cluster is now healthy 2024-02-18T01:17:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:01 smithi105 bash[27669]: cluster 2024-02-18T01:17:01.604857+0000 mon.a (mon.0) 528 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-18T01:17:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:01 smithi105 bash[27669]: cluster 2024-02-18T01:17:01.604942+0000 mon.a (mon.0) 529 : cluster [INF] Cluster is now healthy 2024-02-18T01:17:01.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:01 smithi155 bash[20805]: cluster 2024-02-18T01:17:01.604857+0000 mon.a (mon.0) 528 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-02-18T01:17:01.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:01 smithi155 bash[20805]: cluster 2024-02-18T01:17:01.604942+0000 mon.a (mon.0) 529 : cluster [INF] Cluster is now healthy 2024-02-18T01:17:02.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:02 smithi105 bash[19734]: cluster 2024-02-18T01:17:01.553585+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:02 smithi105 bash[27669]: cluster 2024-02-18T01:17:01.553585+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:02 smithi155 bash[20805]: cluster 2024-02-18T01:17:01.553585+0000 mgr.y (mgr.14146) 263 : cluster [DBG] pgmap v254: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:03.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- 172.21.15.155:0/577520502 >> v1:172.21.15.155:6789/0 conn(0x7f23340fbe20 legacy=0x7f23340fe210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:03.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- 172.21.15.155:0/577520502 shutdown_connections 2024-02-18T01:17:03.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- 172.21.15.155:0/577520502 >> 172.21.15.155:0/577520502 conn(0x7f23340f1610 msgr2=0x7f23340f3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:03.375 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- 172.21.15.155:0/577520502 shutdown_connections 2024-02-18T01:17:03.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- 172.21.15.155:0/577520502 wait complete. 2024-02-18T01:17:03.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 Processor -- start 2024-02-18T01:17:03.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- start start 2024-02-18T01:17:03.376 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f23341107e0 con 0x7f23340ff930 2024-02-18T01:17:03.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f23341119e0 con 0x7f23340fbe20 2024-02-18T01:17:03.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f233c26d700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2334112be0 con 0x7f23340f5ad0 2024-02-18T01:17:03.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2339808700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f23340fbe20 0x7f23340fe500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:33384/0 (socket says 172.21.15.155:33384) 2024-02-18T01:17:03.377 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2339808700 1 -- 172.21.15.155:0/3074969199 learned_addr learned my addr 172.21.15.155:0/3074969199 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219989925 0 0) 0x7f23341119e0 con 0x7f23340fbe20 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c003030 con 0x7f23340fbe20 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714718471 0 0) 0x7f2334112be0 con 0x7f23340f5ad0 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23341119e0 con 0x7f23340f5ad0 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2940293810 0 0) 0x7f23341107e0 con 0x7f23340ff930 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2334112be0 con 0x7f23340ff930 2024-02-18T01:17:03.378 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1427691104 0 0) 0x7f231c003030 con 0x7f23340fbe20 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f23341107e0 con 0x7f23340fbe20 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2328001430 con 0x7f23340fbe20 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3217473624 0 0) 0x7f23341107e0 con 0x7f23340fbe20 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 >> v1:172.21.15.105:6790/0 conn(0x7f23340f5ad0 legacy=0x7f23340fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 >> v1:172.21.15.105:6789/0 conn(0x7f23340ff930 legacy=0x7f233410ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.372+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2334112dd0 con 0x7f23340fbe20 2024-02-18T01:17:03.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2334113080 con 0x7f23340fbe20 2024-02-18T01:17:03.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2334113570 con 0x7f23340fbe20 2024-02-18T01:17:03.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f23280016a0 con 0x7f23340fbe20 2024-02-18T01:17:03.380 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f23280038e0 con 0x7f23340fbe20 2024-02-18T01:17:03.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2320004f50 con 0x7f23340fbe20 2024-02-18T01:17:03.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f2328003b20 con 0x7f23340fbe20 2024-02-18T01:17:03.381 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.376+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4054+0+0 (unknown 32312736 0 0) 0x7f2328077fb0 con 0x7f23340fbe20 2024-02-18T01:17:03.388 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.384+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f2328048950 con 0x7f23340fbe20 2024-02-18T01:17:03.580 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.576+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi105=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f2320000bc0 con 0x7f231c061cb0 2024-02-18T01:17:03.594 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f2333779700 1 -- 172.21.15.155:0/3074969199 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (unknown 0 0 1123153589) 0x7f2320000bc0 con 0x7f231c061cb0 2024-02-18T01:17:03.594 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled rgw.foo.a update... 2024-02-18T01:17:03.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 >> v1:172.21.15.105:6800/3087430119 conn(0x7f231c061cb0 legacy=0x7f231c064170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:03.596 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 >> v1:172.21.15.155:6789/0 conn(0x7f23340fbe20 legacy=0x7f23340fe500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:03.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 shutdown_connections 2024-02-18T01:17:03.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 >> 172.21.15.155:0/3074969199 conn(0x7f23340f1610 msgr2=0x7f2334102e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:03.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 shutdown_connections 2024-02-18T01:17:03.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:03.592+0000 7f233c26d700 1 -- 172.21.15.155:0/3074969199 wait complete. 2024-02-18T01:17:04.344 DEBUG:teuthology.orchestra.run.smithi105:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@rgw.foo.a.service 2024-02-18T01:17:04.347 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi155 2024-02-18T01:17:04.348 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool create iscsi 3 3 replicated 2024-02-18T01:17:04.422 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:04.593 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:04 smithi155 bash[20805]: cluster 2024-02-18T01:17:03.554147+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[19734]: cluster 2024-02-18T01:17:03.554147+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[19734]: audit 2024-02-18T01:17:03.581799+0000 mgr.y (mgr.14146) 265 : audit [DBG] from='client.24343 v1:172.21.15.155:0/3074969199' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi105=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[19734]: cephadm 2024-02-18T01:17:03.585554+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[19734]: audit 2024-02-18T01:17:03.592504+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[27669]: cluster 2024-02-18T01:17:03.554147+0000 mgr.y (mgr.14146) 264 : cluster [DBG] pgmap v255: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[27669]: audit 2024-02-18T01:17:03.581799+0000 mgr.y (mgr.14146) 265 : audit [DBG] from='client.24343 v1:172.21.15.155:0/3074969199' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi105=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[27669]: cephadm 2024-02-18T01:17:03.585554+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:04.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:04 smithi105 bash[27669]: audit 2024-02-18T01:17:03.592504+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:04.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:04 smithi155 bash[20805]: audit 2024-02-18T01:17:03.581799+0000 mgr.y (mgr.14146) 265 : audit [DBG] from='client.24343 v1:172.21.15.155:0/3074969199' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi105=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:04.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:04 smithi155 bash[20805]: cephadm 2024-02-18T01:17:03.585554+0000 mgr.y (mgr.14146) 266 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:04.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:04 smithi155 bash[20805]: audit 2024-02-18T01:17:03.592504+0000 mon.a (mon.0) 530 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: cephadm 2024-02-18T01:17:05.171527+0000 mgr.y (mgr.14146) 267 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.181993+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.184717+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.186178+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.187573+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.189042+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: cephadm 2024-02-18T01:17:05.190212+0000 mgr.y (mgr.14146) 268 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4329M 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.199805+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.201586+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.203110+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.213169+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: cephadm 2024-02-18T01:17:05.217538+0000 mgr.y (mgr.14146) 269 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.224913+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.225797+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T01:17:06.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.233624+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T01:17:06.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.241630+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: audit 2024-02-18T01:17:05.243063+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: cephadm 2024-02-18T01:17:05.244281+0000 mgr.y (mgr.14146) 270 : cephadm [INF] Deploying daemon rgw.foo.a on smithi105 2024-02-18T01:17:06.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:06 smithi155 bash[20805]: cluster 2024-02-18T01:17:05.554788+0000 mgr.y (mgr.14146) 271 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:06.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: cephadm 2024-02-18T01:17:05.171527+0000 mgr.y (mgr.14146) 267 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:17:06.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.181993+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.184717+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.186178+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.187573+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.189042+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: cephadm 2024-02-18T01:17:05.190212+0000 mgr.y (mgr.14146) 268 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4329M 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.199805+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.201586+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.203110+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.213169+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: cephadm 2024-02-18T01:17:05.217538+0000 mgr.y (mgr.14146) 269 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.224913+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.225797+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.233624+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.241630+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: audit 2024-02-18T01:17:05.243063+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: cephadm 2024-02-18T01:17:05.244281+0000 mgr.y (mgr.14146) 270 : cephadm [INF] Deploying daemon rgw.foo.a on smithi105 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[19734]: cluster 2024-02-18T01:17:05.554788+0000 mgr.y (mgr.14146) 271 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: cephadm 2024-02-18T01:17:05.171527+0000 mgr.y (mgr.14146) 267 : cephadm [INF] Detected new or changed devices on smithi155 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.181993+0000 mon.a (mon.0) 531 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.184717+0000 mon.a (mon.0) 532 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.186178+0000 mon.a (mon.0) 533 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.187573+0000 mon.a (mon.0) 534 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.189042+0000 mon.a (mon.0) 535 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:06.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: cephadm 2024-02-18T01:17:05.190212+0000 mgr.y (mgr.14146) 268 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4329M 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.199805+0000 mon.a (mon.0) 536 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.201586+0000 mon.a (mon.0) 537 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.203110+0000 mon.a (mon.0) 538 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.213169+0000 mon.a (mon.0) 539 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: cephadm 2024-02-18T01:17:05.217538+0000 mgr.y (mgr.14146) 269 : cephadm [INF] Saving service rgw.foo.a spec with placement smithi105=foo.a;count:1 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.224913+0000 mon.a (mon.0) 540 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.225797+0000 mon.a (mon.0) 541 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.233624+0000 mon.a (mon.0) 542 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.241630+0000 mon.a (mon.0) 543 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: audit 2024-02-18T01:17:05.243063+0000 mon.a (mon.0) 544 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: cephadm 2024-02-18T01:17:05.244281+0000 mgr.y (mgr.14146) 270 : cephadm [INF] Deploying daemon rgw.foo.a on smithi105 2024-02-18T01:17:06.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:06 smithi105 bash[27669]: cluster 2024-02-18T01:17:05.554788+0000 mgr.y (mgr.14146) 271 : cluster [DBG] pgmap v256: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:07.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- 172.21.15.155:0/282568295 >> v1:172.21.15.155:6789/0 conn(0x7fc9740fbcf0 legacy=0x7fc9740fe0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:07.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- 172.21.15.155:0/282568295 shutdown_connections 2024-02-18T01:17:07.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- 172.21.15.155:0/282568295 >> 172.21.15.155:0/282568295 conn(0x7fc9740f2f80 msgr2=0x7fc9740f53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:07.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- 172.21.15.155:0/282568295 shutdown_connections 2024-02-18T01:17:07.653 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- 172.21.15.155:0/282568295 wait complete. 2024-02-18T01:17:07.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 Processor -- start 2024-02-18T01:17:07.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- start start 2024-02-18T01:17:07.654 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc974114c30 con 0x7fc9740ff920 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc974115e30 con 0x7fc9740f7820 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97c5cb700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc974117030 con 0x7fc9740fbcf0 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97a367700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc9740f7820 0x7fc9740fde00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:33398/0 (socket says 172.21.15.155:33398) 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.648+0000 7fc97a367700 1 -- 172.21.15.155:0/3785384481 learned_addr learned my addr 172.21.15.155:0/3785384481 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489914799 0 0) 0x7fc974115e30 con 0x7fc9740f7820 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc960003630 con 0x7fc9740f7820 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871761903 0 0) 0x7fc974117030 con 0x7fc9740fbcf0 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974115e30 con 0x7fc9740fbcf0 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3005400697 0 0) 0x7fc974114c30 con 0x7fc9740ff920 2024-02-18T01:17:07.655 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc974117030 con 0x7fc9740ff920 2024-02-18T01:17:07.657 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 822594152 0 0) 0x7fc974115e30 con 0x7fc9740fbcf0 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc974114c30 con 0x7fc9740fbcf0 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc968001ec0 con 0x7fc9740fbcf0 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2654924859 0 0) 0x7fc960003630 con 0x7fc9740f7820 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc974115e30 con 0x7fc9740f7820 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc9700018d0 con 0x7fc9740f7820 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1507287987 0 0) 0x7fc974117030 con 0x7fc9740ff920 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc960003630 con 0x7fc9740ff920 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc96c001c40 con 0x7fc9740ff920 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2904233336 0 0) 0x7fc974114c30 con 0x7fc9740fbcf0 2024-02-18T01:17:07.658 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 >> v1:172.21.15.155:6789/0 conn(0x7fc9740f7820 legacy=0x7fc9740fde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 >> v1:172.21.15.105:6789/0 conn(0x7fc9740ff920 legacy=0x7fc9741133b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc974118230 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc974114e20 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc974115310 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fc968001bb0 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc968003650 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc958004f20 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.652+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7fc9680020c0 con 0x7fc9740fbcf0 2024-02-18T01:17:07.659 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.656+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(51..51 src has 1..51) v4 ==== 4054+0+0 (unknown 32312736 0 0) 0x7fc968075750 con 0x7fc9740fbcf0 2024-02-18T01:17:07.669 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.664+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc968046090 con 0x7fc9740fbcf0 2024-02-18T01:17:07.853 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:07.848+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fc958005750 con 0x7fc9740fbcf0 2024-02-18T01:17:07.989 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:Feb 18 01:17:07 smithi105 systemd[1]: Started Ceph rgw.foo.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:17:08.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.683+0000 7fc9677fe700 1 -- 172.21.15.155:0/3785384481 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'iscsi' created v52) v1 ==== 154+0+0 (unknown 111238501 0 0) 0x7fc968047fc0 con 0x7fc9740fbcf0 2024-02-18T01:17:08.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 >> v1:172.21.15.105:6800/3087430119 conn(0x7fc960062240 legacy=0x7fc960064700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:08.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 >> v1:172.21.15.105:6790/0 conn(0x7fc9740fbcf0 legacy=0x7fc9740fe510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:08.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 shutdown_connections 2024-02-18T01:17:08.692 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 >> 172.21.15.155:0/3785384481 conn(0x7fc9740f2f80 msgr2=0x7fc97406af10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:08.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 shutdown_connections 2024-02-18T01:17:08.693 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:08.687+0000 7fc97c5cb700 1 -- 172.21.15.155:0/3785384481 wait complete. 2024-02-18T01:17:08.694 INFO:teuthology.orchestra.run.smithi155.stderr:pool 'iscsi' created 2024-02-18T01:17:08.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[27669]: cluster 2024-02-18T01:17:07.555454+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[27669]: audit 2024-02-18T01:17:07.682150+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[27669]: audit 2024-02-18T01:17:07.683811+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[27669]: audit 2024-02-18T01:17:07.852826+0000 mon.c (mon.2) 18 : audit [INF] from='client.? v1:172.21.15.155:0/3785384481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[27669]: audit 2024-02-18T01:17:07.853419+0000 mon.a (mon.0) 547 : audit [INF] from='client.24353 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[19734]: cluster 2024-02-18T01:17:07.555454+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[19734]: audit 2024-02-18T01:17:07.682150+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[19734]: audit 2024-02-18T01:17:07.683811+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[19734]: audit 2024-02-18T01:17:07.852826+0000 mon.c (mon.2) 18 : audit [INF] from='client.? v1:172.21.15.155:0/3785384481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:08 smithi105 bash[19734]: audit 2024-02-18T01:17:07.853419+0000 mon.a (mon.0) 547 : audit [INF] from='client.24353 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:08 smithi155 bash[20805]: cluster 2024-02-18T01:17:07.555454+0000 mgr.y (mgr.14146) 272 : cluster [DBG] pgmap v257: 1 pgs: 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:08 smithi155 bash[20805]: audit 2024-02-18T01:17:07.682150+0000 mon.a (mon.0) 545 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:08 smithi155 bash[20805]: audit 2024-02-18T01:17:07.683811+0000 mon.a (mon.0) 546 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:08 smithi155 bash[20805]: audit 2024-02-18T01:17:07.852826+0000 mon.c (mon.2) 18 : audit [INF] from='client.? v1:172.21.15.155:0/3785384481' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:08 smithi155 bash[20805]: audit 2024-02-18T01:17:07.853419+0000 mon.a (mon.0) 547 : audit [INF] from='client.24353 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-02-18T01:17:09.317 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool application enable iscsi rbd 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[19734]: audit 2024-02-18T01:17:08.687379+0000 mon.a (mon.0) 548 : audit [INF] from='client.24353 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[19734]: cluster 2024-02-18T01:17:08.687590+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[19734]: cluster 2024-02-18T01:17:09.697469+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[27669]: audit 2024-02-18T01:17:08.687379+0000 mon.a (mon.0) 548 : audit [INF] from='client.24353 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[27669]: cluster 2024-02-18T01:17:08.687590+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-18T01:17:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:09 smithi105 bash[27669]: cluster 2024-02-18T01:17:09.697469+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T01:17:10.101 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:09 smithi155 bash[20805]: audit 2024-02-18T01:17:08.687379+0000 mon.a (mon.0) 548 : audit [INF] from='client.24353 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "iscsi", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-02-18T01:17:10.101 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:09 smithi155 bash[20805]: cluster 2024-02-18T01:17:08.687590+0000 mon.a (mon.0) 549 : cluster [DBG] osdmap e52: 8 total, 8 up, 8 in 2024-02-18T01:17:10.101 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:09 smithi155 bash[20805]: cluster 2024-02-18T01:17:09.697469+0000 mon.a (mon.0) 550 : cluster [DBG] osdmap e53: 8 total, 8 up, 8 in 2024-02-18T01:17:10.786 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:10 smithi155 bash[20805]: cluster 2024-02-18T01:17:09.556178+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v259: 4 pgs: 3 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:10.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:10 smithi155 bash[20805]: audit 2024-02-18T01:17:09.699762+0000 mon.a (mon.0) 551 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T01:17:10.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:10 smithi155 bash[20805]: audit 2024-02-18T01:17:10.698475+0000 mon.a (mon.0) 552 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T01:17:10.787 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:10 smithi155 bash[20805]: cluster 2024-02-18T01:17:10.698829+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[19734]: cluster 2024-02-18T01:17:09.556178+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v259: 4 pgs: 3 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[19734]: audit 2024-02-18T01:17:09.699762+0000 mon.a (mon.0) 551 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[19734]: audit 2024-02-18T01:17:10.698475+0000 mon.a (mon.0) 552 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[19734]: cluster 2024-02-18T01:17:10.698829+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[27669]: cluster 2024-02-18T01:17:09.556178+0000 mgr.y (mgr.14146) 273 : cluster [DBG] pgmap v259: 4 pgs: 3 unknown, 1 active+clean; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[27669]: audit 2024-02-18T01:17:09.699762+0000 mon.a (mon.0) 551 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[27669]: audit 2024-02-18T01:17:10.698475+0000 mon.a (mon.0) 552 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-02-18T01:17:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:10 smithi105 bash[27669]: cluster 2024-02-18T01:17:10.698829+0000 mon.a (mon.0) 553 : cluster [DBG] osdmap e54: 8 total, 8 up, 8 in 2024-02-18T01:17:12.303 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- 172.21.15.155:0/3819668970 >> v1:172.21.15.155:6789/0 conn(0x7f9edc0f7d00 legacy=0x7f9edc0f8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:12.303 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- 172.21.15.155:0/3819668970 shutdown_connections 2024-02-18T01:17:12.304 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- 172.21.15.155:0/3819668970 >> 172.21.15.155:0/3819668970 conn(0x7f9edc0f1630 msgr2=0x7f9edc0f3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:12.304 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- 172.21.15.155:0/3819668970 shutdown_connections 2024-02-18T01:17:12.304 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- 172.21.15.155:0/3819668970 wait complete. 2024-02-18T01:17:12.304 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 Processor -- start 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- start start 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9edc10a760 con 0x7f9edc0f7d00 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9edc115fc0 con 0x7f9edc0f6940 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee3090700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9edc1171a0 con 0x7f9edc0f74e0 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee0e2c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f9edc0f6940 0x7f9edc109b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:42910/0 (socket says 172.21.15.155:42910) 2024-02-18T01:17:12.305 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ee0e2c700 1 -- 172.21.15.155:0/3100026139 learned_addr learned my addr 172.21.15.155:0/3100026139 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:12.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 402192963 0 0) 0x7f9edc115fc0 con 0x7f9edc0f6940 2024-02-18T01:17:12.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ec8003030 con 0x7f9edc0f6940 2024-02-18T01:17:12.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166214923 0 0) 0x7f9edc1171a0 con 0x7f9edc0f74e0 2024-02-18T01:17:12.306 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc115fc0 con 0x7f9edc0f74e0 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2509309581 0 0) 0x7f9edc10a760 con 0x7f9edc0f7d00 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9edc1171a0 con 0x7f9edc0f7d00 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2768593794 0 0) 0x7f9ec8003030 con 0x7f9edc0f6940 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9edc10a760 con 0x7f9edc0f6940 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.299+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f9ed4001950 con 0x7f9edc0f6940 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 554255808 0 0) 0x7f9edc115fc0 con 0x7f9edc0f74e0 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9ec8003030 con 0x7f9edc0f74e0 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f9ec4001880 con 0x7f9edc0f74e0 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 431544870 0 0) 0x7f9edc1171a0 con 0x7f9edc0f7d00 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9edc115fc0 con 0x7f9edc0f7d00 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2136778014 0 0) 0x7f9edc10a760 con 0x7f9edc0f6940 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 >> v1:172.21.15.105:6790/0 conn(0x7f9edc0f74e0 legacy=0x7f9edc110160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 >> v1:172.21.15.105:6789/0 conn(0x7f9edc0f7d00 legacy=0x7f9edc113890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:12.307 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9edc118380 con 0x7f9edc0f6940 2024-02-18T01:17:12.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9edc117390 con 0x7f9edc0f6940 2024-02-18T01:17:12.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f9edc117880 con 0x7f9edc0f6940 2024-02-18T01:17:12.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f9ed4001f70 con 0x7f9edc0f6940 2024-02-18T01:17:12.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f9ed4003db0 con 0x7f9edc0f6940 2024-02-18T01:17:12.308 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9edc047540 con 0x7f9edc0f6940 2024-02-18T01:17:12.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7f9ed4003fb0 con 0x7f9edc0f6940 2024-02-18T01:17:12.309 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.303+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(55..55 src has 1..55) v4 ==== 5111+0+0 (unknown 2608852310 0 0) 0x7f9ed40786f0 con 0x7f9edc0f6940 2024-02-18T01:17:12.318 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.315+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f9ed4048c70 con 0x7f9edc0f6940 2024-02-18T01:17:12.508 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.503+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"} v 0) v1 -- 0x7f9edc106860 con 0x7f9edc0f6940 2024-02-18T01:17:12.821 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.815+0000 7f9ed9ffb700 1 -- 172.21.15.155:0/3100026139 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]=0 enabled application 'rbd' on pool 'iscsi' v56) v1 ==== 147+0+0 (unknown 4288338289 0 0) 0x7f9ed404aba0 con 0x7f9edc0f6940 2024-02-18T01:17:12.825 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.819+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 >> v1:172.21.15.105:6800/3087430119 conn(0x7f9ec80622d0 legacy=0x7f9ec8064790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:12.826 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.819+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 >> v1:172.21.15.155:6789/0 conn(0x7f9edc0f6940 legacy=0x7f9edc109b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:12.826 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.819+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 shutdown_connections 2024-02-18T01:17:12.826 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.819+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 >> 172.21.15.155:0/3100026139 conn(0x7f9edc0f1630 msgr2=0x7f9edc0f3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:12.827 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.823+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 shutdown_connections 2024-02-18T01:17:12.827 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:12.823+0000 7f9ee3090700 1 -- 172.21.15.155:0/3100026139 wait complete. 2024-02-18T01:17:12.830 INFO:teuthology.orchestra.run.smithi155.stderr:enabled application 'rbd' on pool 'iscsi' 2024-02-18T01:17:13.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: cluster 2024-02-18T01:17:11.556806+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v262: 36 pgs: 9 active+clean, 3 creating+activating, 13 creating+peering, 11 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: cluster 2024-02-18T01:17:11.804376+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: audit 2024-02-18T01:17:11.806066+0000 mon.a (mon.0) 555 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: audit 2024-02-18T01:17:12.507962+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.155:0/3100026139' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: audit 2024-02-18T01:17:12.508779+0000 mon.a (mon.0) 556 : audit [INF] from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[19734]: audit 2024-02-18T01:17:12.552730+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: cluster 2024-02-18T01:17:11.556806+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v262: 36 pgs: 9 active+clean, 3 creating+activating, 13 creating+peering, 11 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: cluster 2024-02-18T01:17:11.804376+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T01:17:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: audit 2024-02-18T01:17:11.806066+0000 mon.a (mon.0) 555 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T01:17:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: audit 2024-02-18T01:17:12.507962+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.155:0/3100026139' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: audit 2024-02-18T01:17:12.508779+0000 mon.a (mon.0) 556 : audit [INF] from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:12 smithi105 bash[27669]: audit 2024-02-18T01:17:12.552730+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: cluster 2024-02-18T01:17:11.556806+0000 mgr.y (mgr.14146) 274 : cluster [DBG] pgmap v262: 36 pgs: 9 active+clean, 3 creating+activating, 13 creating+peering, 11 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: cluster 2024-02-18T01:17:11.804376+0000 mon.a (mon.0) 554 : cluster [DBG] osdmap e55: 8 total, 8 up, 8 in 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: audit 2024-02-18T01:17:11.806066+0000 mon.a (mon.0) 555 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: audit 2024-02-18T01:17:12.507962+0000 mon.b (mon.1) 19 : audit [INF] from='client.? v1:172.21.15.155:0/3100026139' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: audit 2024-02-18T01:17:12.508779+0000 mon.a (mon.0) 556 : audit [INF] from='client.24364 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]: dispatch 2024-02-18T01:17:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:12 smithi155 bash[20805]: audit 2024-02-18T01:17:12.552730+0000 mon.a (mon.0) 557 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:13.529 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply iscsi iscsi user password --placement '1;smithi155=iscsi.a' 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[27669]: audit 2024-02-18T01:17:12.809278+0000 mon.a (mon.0) 558 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[27669]: audit 2024-02-18T01:17:12.809920+0000 mon.a (mon.0) 559 : audit [INF] from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[27669]: cluster 2024-02-18T01:17:12.810129+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[27669]: cluster 2024-02-18T01:17:13.813706+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[19734]: audit 2024-02-18T01:17:12.809278+0000 mon.a (mon.0) 558 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[19734]: audit 2024-02-18T01:17:12.809920+0000 mon.a (mon.0) 559 : audit [INF] from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[19734]: cluster 2024-02-18T01:17:12.810129+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T01:17:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:13 smithi105 bash[19734]: cluster 2024-02-18T01:17:13.813706+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T01:17:14.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:13 smithi155 bash[20805]: audit 2024-02-18T01:17:12.809278+0000 mon.a (mon.0) 558 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-02-18T01:17:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:13 smithi155 bash[20805]: audit 2024-02-18T01:17:12.809920+0000 mon.a (mon.0) 559 : audit [INF] from='client.24364 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "iscsi", "app": "rbd"}]': finished 2024-02-18T01:17:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:13 smithi155 bash[20805]: cluster 2024-02-18T01:17:12.810129+0000 mon.a (mon.0) 560 : cluster [DBG] osdmap e56: 8 total, 8 up, 8 in 2024-02-18T01:17:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:13 smithi155 bash[20805]: cluster 2024-02-18T01:17:13.813706+0000 mon.a (mon.0) 561 : cluster [DBG] osdmap e57: 8 total, 8 up, 8 in 2024-02-18T01:17:14.990 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: cluster 2024-02-18T01:17:13.557432+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v265: 68 pgs: 20 active+clean, 3 creating+activating, 18 creating+peering, 27 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: audit 2024-02-18T01:17:13.818003+0000 mon.a (mon.0) 562 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: audit 2024-02-18T01:17:14.116715+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: audit 2024-02-18T01:17:14.390192+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: audit 2024-02-18T01:17:14.391554+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: audit 2024-02-18T01:17:14.816209+0000 mon.a (mon.0) 566 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T01:17:14.991 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:14 smithi155 bash[20805]: cluster 2024-02-18T01:17:14.816732+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: cluster 2024-02-18T01:17:13.557432+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v265: 68 pgs: 20 active+clean, 3 creating+activating, 18 creating+peering, 27 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: audit 2024-02-18T01:17:13.818003+0000 mon.a (mon.0) 562 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: audit 2024-02-18T01:17:14.116715+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: audit 2024-02-18T01:17:14.390192+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: audit 2024-02-18T01:17:14.391554+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: audit 2024-02-18T01:17:14.816209+0000 mon.a (mon.0) 566 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[19734]: cluster 2024-02-18T01:17:14.816732+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T01:17:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: cluster 2024-02-18T01:17:13.557432+0000 mgr.y (mgr.14146) 275 : cluster [DBG] pgmap v265: 68 pgs: 20 active+clean, 3 creating+activating, 18 creating+peering, 27 unknown; 0 B data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: audit 2024-02-18T01:17:13.818003+0000 mon.a (mon.0) 562 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: audit 2024-02-18T01:17:14.116715+0000 mon.a (mon.0) 563 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: audit 2024-02-18T01:17:14.390192+0000 mon.a (mon.0) 564 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: audit 2024-02-18T01:17:14.391554+0000 mon.a (mon.0) 565 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: audit 2024-02-18T01:17:14.816209+0000 mon.a (mon.0) 566 : audit [INF] from='client.? v1:172.21.15.105:0/2914114458' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-02-18T01:17:15.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:14 smithi105 bash[27669]: cluster 2024-02-18T01:17:14.816732+0000 mon.a (mon.0) 567 : cluster [DBG] osdmap e58: 8 total, 8 up, 8 in 2024-02-18T01:17:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:15 smithi105 bash[19734]: cephadm 2024-02-18T01:17:14.401828+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T01:17:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:15 smithi105 bash[19734]: cluster 2024-02-18T01:17:15.824235+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T01:17:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:15 smithi105 bash[27669]: cephadm 2024-02-18T01:17:14.401828+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T01:17:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:15 smithi105 bash[27669]: cluster 2024-02-18T01:17:15.824235+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T01:17:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:15 smithi155 bash[20805]: cephadm 2024-02-18T01:17:14.401828+0000 mgr.y (mgr.14146) 276 : cephadm [INF] Checking dashboard <-> RGW credentials 2024-02-18T01:17:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:15 smithi155 bash[20805]: cluster 2024-02-18T01:17:15.824235+0000 mon.a (mon.0) 568 : cluster [DBG] osdmap e59: 8 total, 8 up, 8 in 2024-02-18T01:17:16.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.679+0000 7fc618fed700 1 -- 172.21.15.155:0/1024534632 >> v1:172.21.15.155:6789/0 conn(0x7fc6140fe8d0 legacy=0x7fc614100d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:16.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.679+0000 7fc618fed700 1 -- 172.21.15.155:0/1024534632 shutdown_connections 2024-02-18T01:17:16.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.679+0000 7fc618fed700 1 -- 172.21.15.155:0/1024534632 >> 172.21.15.155:0/1024534632 conn(0x7fc6140f16b0 msgr2=0x7fc6140f3ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:16.685 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.679+0000 7fc618fed700 1 -- 172.21.15.155:0/1024534632 shutdown_connections 2024-02-18T01:17:16.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.679+0000 7fc618fed700 1 -- 172.21.15.155:0/1024534632 wait complete. 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 Processor -- start 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- start start 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6140f69e0 con 0x7fc6140fadc0 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc614113de0 con 0x7fc6140f6d40 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc61259c700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fc6140f6d40 0x7fc61410a7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:42932/0 (socket says 172.21.15.155:42932) 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc61259c700 1 -- 172.21.15.155:0/3007918620 learned_addr learned my addr 172.21.15.155:0/3007918620 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc614114fe0 con 0x7fc6140fe8d0 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 842326420 0 0) 0x7fc614113de0 con 0x7fc6140f6d40 2024-02-18T01:17:16.687 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5fc003030 con 0x7fc6140f6d40 2024-02-18T01:17:16.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873972137 0 0) 0x7fc614114fe0 con 0x7fc6140fe8d0 2024-02-18T01:17:16.688 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc614113de0 con 0x7fc6140fe8d0 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3475841417 0 0) 0x7fc5fc003030 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc614114fe0 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc604001980 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 165205718 0 0) 0x7fc614102640 con 0x7fc6140fadc0 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5fc003030 con 0x7fc6140fadc0 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 985153213 0 0) 0x7fc614114fe0 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 >> v1:172.21.15.105:6790/0 conn(0x7fc6140fe8d0 legacy=0x7fc614111650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 >> v1:172.21.15.105:6789/0 conn(0x7fc6140fadc0 legacy=0x7fc61410df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6141161e0 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6141151d0 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6141156c0 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fc604001d10 con 0x7fc6140f6d40 2024-02-18T01:17:16.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc604005ca0 con 0x7fc6140f6d40 2024-02-18T01:17:16.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.683+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f4004f20 con 0x7fc6140f6d40 2024-02-18T01:17:16.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.687+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 14) v1 ==== 77030+0+0 (unknown 980597065 0 0) 0x7fc604002970 con 0x7fc6140f6d40 2024-02-18T01:17:16.690 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.687+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(59..59 src has 1..59) v4 ==== 5857+0+0 (unknown 1031060645 0 0) 0x7fc604078470 con 0x7fc6140f6d40 2024-02-18T01:17:16.699 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.695+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc6040485e0 con 0x7fc6140f6d40 2024-02-18T01:17:16.904 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.899+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5f4000c40 con 0x7fc5fc062160 2024-02-18T01:17:16.920 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.915+0000 7fc6037fe700 1 -- 172.21.15.155:0/3007918620 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (unknown 0 0 3282307126) 0x7fc5f4000c40 con 0x7fc5fc062160 2024-02-18T01:17:16.921 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled iscsi.iscsi update... 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 >> v1:172.21.15.105:6800/3087430119 conn(0x7fc5fc062160 legacy=0x7fc5fc064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 >> v1:172.21.15.155:6789/0 conn(0x7fc6140f6d40 legacy=0x7fc61410a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 shutdown_connections 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 >> 172.21.15.155:0/3007918620 conn(0x7fc6140f16b0 msgr2=0x7fc6140fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 shutdown_connections 2024-02-18T01:17:16.923 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:16.919+0000 7fc618fed700 1 -- 172.21.15.155:0/3007918620 wait complete. 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: cluster 2024-02-18T01:17:15.558440+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v268: 100 pgs: 54 active+clean, 3 creating+activating, 13 creating+peering, 30 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:15.831591+0000 mon.c (mon.2) 19 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:15.831883+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:15.836760+0000 mon.a (mon.0) 569 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:15.837062+0000 mon.a (mon.0) 570 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.827014+0000 mon.a (mon.0) 571 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.827186+0000 mon.a (mon.0) 572 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: cluster 2024-02-18T01:17:16.827309+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.833630+0000 mon.c (mon.2) 21 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.834840+0000 mon.c (mon.2) 22 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.835897+0000 mon.a (mon.0) 574 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[19734]: audit 2024-02-18T01:17:16.836201+0000 mon.a (mon.0) 575 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: cluster 2024-02-18T01:17:15.558440+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v268: 100 pgs: 54 active+clean, 3 creating+activating, 13 creating+peering, 30 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:15.831591+0000 mon.c (mon.2) 19 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:15.831883+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:15.836760+0000 mon.a (mon.0) 569 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:15.837062+0000 mon.a (mon.0) 570 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.827014+0000 mon.a (mon.0) 571 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.827186+0000 mon.a (mon.0) 572 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: cluster 2024-02-18T01:17:16.827309+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.833630+0000 mon.c (mon.2) 21 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.834840+0000 mon.c (mon.2) 22 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.835897+0000 mon.a (mon.0) 574 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:16 smithi105 bash[27669]: audit 2024-02-18T01:17:16.836201+0000 mon.a (mon.0) 575 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: cluster 2024-02-18T01:17:15.558440+0000 mgr.y (mgr.14146) 277 : cluster [DBG] pgmap v268: 100 pgs: 54 active+clean, 3 creating+activating, 13 creating+peering, 30 unknown; 450 B data, 2.3 GiB used, 713 GiB / 715 GiB avail; 8.0 KiB/s rd, 2.7 KiB/s wr, 10 op/s 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:15.831591+0000 mon.c (mon.2) 19 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:15.831883+0000 mon.c (mon.2) 20 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:15.836760+0000 mon.a (mon.0) 569 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:15.837062+0000 mon.a (mon.0) 570 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.827014+0000 mon.a (mon.0) 571 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.827186+0000 mon.a (mon.0) 572 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-02-18T01:17:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: cluster 2024-02-18T01:17:16.827309+0000 mon.a (mon.0) 573 : cluster [DBG] osdmap e60: 8 total, 8 up, 8 in 2024-02-18T01:17:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.833630+0000 mon.c (mon.2) 21 : audit [INF] from='client.? v1:172.21.15.105:0/2475373584' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.834840+0000 mon.c (mon.2) 22 : audit [INF] from='client.? v1:172.21.15.105:0/1821981542' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.835897+0000 mon.a (mon.0) 574 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:16 smithi155 bash[20805]: audit 2024-02-18T01:17:16.836201+0000 mon.a (mon.0) 575 : audit [INF] from='client.24401 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-02-18T01:17:17.589 DEBUG:teuthology.orchestra.run.smithi155:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@iscsi.iscsi.a.service 2024-02-18T01:17:17.592 INFO:tasks.cephadm:Adding prometheus.a on smithi155 2024-02-18T01:17:17.592 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply prometheus '1;smithi155=a' 2024-02-18T01:17:17.779 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:17.923 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: audit 2024-02-18T01:17:16.906104+0000 mgr.y (mgr.14146) 278 : audit [DBG] from='client.24406 v1:172.21.15.155:0/3007918620' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:17.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: cephadm 2024-02-18T01:17:16.912227+0000 mgr.y (mgr.14146) 279 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi155=iscsi.a;count:1 2024-02-18T01:17:17.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: audit 2024-02-18T01:17:16.918969+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:17.924 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: audit 2024-02-18T01:17:17.834335+0000 mon.a (mon.0) 577 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: audit 2024-02-18T01:17:16.906104+0000 mgr.y (mgr.14146) 278 : audit [DBG] from='client.24406 v1:172.21.15.155:0/3007918620' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: cephadm 2024-02-18T01:17:16.912227+0000 mgr.y (mgr.14146) 279 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi155=iscsi.a;count:1 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: audit 2024-02-18T01:17:16.918969+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: audit 2024-02-18T01:17:17.834335+0000 mon.a (mon.0) 577 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: audit 2024-02-18T01:17:17.834473+0000 mon.a (mon.0) 578 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[19734]: cluster 2024-02-18T01:17:17.834590+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T01:17:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: audit 2024-02-18T01:17:16.906104+0000 mgr.y (mgr.14146) 278 : audit [DBG] from='client.24406 v1:172.21.15.155:0/3007918620' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "iscsi", "api_user": "user", "api_password": "password", "placement": "1;smithi155=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: cephadm 2024-02-18T01:17:16.912227+0000 mgr.y (mgr.14146) 279 : cephadm [INF] Saving service iscsi.iscsi spec with placement smithi155=iscsi.a;count:1 2024-02-18T01:17:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: audit 2024-02-18T01:17:16.918969+0000 mon.a (mon.0) 576 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: audit 2024-02-18T01:17:17.834335+0000 mon.a (mon.0) 577 : audit [INF] from='client.24398 ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: audit 2024-02-18T01:17:17.834473+0000 mon.a (mon.0) 578 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:17 smithi105 bash[27669]: cluster 2024-02-18T01:17:17.834590+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T01:17:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: audit 2024-02-18T01:17:17.834473+0000 mon.a (mon.0) 578 : audit [INF] from='client.24401 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-02-18T01:17:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:17 smithi155 bash[20805]: cluster 2024-02-18T01:17:17.834590+0000 mon.a (mon.0) 579 : cluster [DBG] osdmap e61: 8 total, 8 up, 8 in 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: cluster 2024-02-18T01:17:17.559163+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v271: 132 pgs: 78 active+clean, 23 creating+peering, 31 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.027412+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.034076+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.040756+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: cephadm 2024-02-18T01:17:18.048022+0000 mgr.y (mgr.14146) 281 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.060349+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.064115+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: audit 2024-02-18T01:17:18.076076+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:19.358 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:19 smithi155 bash[20805]: cephadm 2024-02-18T01:17:18.076971+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi155 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: cluster 2024-02-18T01:17:17.559163+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v271: 132 pgs: 78 active+clean, 23 creating+peering, 31 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.027412+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.034076+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.040756+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: cephadm 2024-02-18T01:17:18.048022+0000 mgr.y (mgr.14146) 281 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.060349+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.064115+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: audit 2024-02-18T01:17:18.076076+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[19734]: cephadm 2024-02-18T01:17:18.076971+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi155 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: cluster 2024-02-18T01:17:17.559163+0000 mgr.y (mgr.14146) 280 : cluster [DBG] pgmap v271: 132 pgs: 78 active+clean, 23 creating+peering, 31 unknown; 1.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.6 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.027412+0000 mon.a (mon.0) 580 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.034076+0000 mon.a (mon.0) 581 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.040756+0000 mon.a (mon.0) 582 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: cephadm 2024-02-18T01:17:18.048022+0000 mgr.y (mgr.14146) 281 : cephadm [INF] Checking pool "iscsi" exists for service iscsi.iscsi 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.060349+0000 mon.a (mon.0) 583 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.064115+0000 mon.a (mon.0) 584 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: audit 2024-02-18T01:17:18.076076+0000 mon.a (mon.0) 585 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:19.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:19 smithi105 bash[27669]: cephadm 2024-02-18T01:17:18.076971+0000 mgr.y (mgr.14146) 282 : cephadm [INF] Deploying daemon iscsi.iscsi.a on smithi155 2024-02-18T01:17:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:20 smithi105 bash[19734]: cluster 2024-02-18T01:17:19.076126+0000 mon.a (mon.0) 586 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-18T01:17:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:20 smithi105 bash[27669]: cluster 2024-02-18T01:17:19.076126+0000 mon.a (mon.0) 586 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-18T01:17:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:20 smithi155 bash[20805]: cluster 2024-02-18T01:17:19.076126+0000 mon.a (mon.0) 586 : cluster [DBG] mgrmap e15: y(active, since 7m), standbys: x 2024-02-18T01:17:21.114 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:21 smithi155 bash[20805]: cluster 2024-02-18T01:17:19.560325+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v273: 132 pgs: 102 active+clean, 15 creating+peering, 15 unknown; 1.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 53 op/s 2024-02-18T01:17:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:21 smithi105 bash[27669]: cluster 2024-02-18T01:17:19.560325+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v273: 132 pgs: 102 active+clean, 15 creating+peering, 15 unknown; 1.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 53 op/s 2024-02-18T01:17:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:21 smithi105 bash[19734]: cluster 2024-02-18T01:17:19.560325+0000 mgr.y (mgr.14146) 283 : cluster [DBG] pgmap v273: 132 pgs: 102 active+clean, 15 creating+peering, 15 unknown; 1.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 2.1 KiB/s wr, 53 op/s 2024-02-18T01:17:21.494 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:21 smithi155 systemd[1]: Started Ceph iscsi.iscsi.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:17:22.363 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- 172.21.15.155:0/1755814127 >> v1:172.21.15.155:6789/0 conn(0x7faf380f9a30 legacy=0x7faf380f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- 172.21.15.155:0/1755814127 shutdown_connections 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- 172.21.15.155:0/1755814127 >> 172.21.15.155:0/1755814127 conn(0x7faf380f54b0 msgr2=0x7faf380f78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- 172.21.15.155:0/1755814127 shutdown_connections 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- 172.21.15.155:0/1755814127 wait complete. 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 Processor -- start 2024-02-18T01:17:22.364 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- start start 2024-02-18T01:17:22.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf3810a600 con 0x7faf380f9a30 2024-02-18T01:17:22.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf38115e60 con 0x7faf38101b30 2024-02-18T01:17:22.365 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3cf22700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7faf38117040 con 0x7faf380fdf00 2024-02-18T01:17:22.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf35d9b700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7faf380fdf00 0x7faf38110000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.155:42824/0 (socket says 172.21.15.155:42824) 2024-02-18T01:17:22.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf3659c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7faf380f9a30 0x7faf38109a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.155:59788/0 (socket says 172.21.15.155:59788) 2024-02-18T01:17:22.366 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf35d9b700 1 -- 172.21.15.155:0/3463448869 learned_addr learned my addr 172.21.15.155:0/3463448869 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.359+0000 7faf36d9d700 1 --1- 172.21.15.155:0/3463448869 >> v1:172.21.15.155:6789/0 conn(0x7faf38101b30 0x7faf38113730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:56630/0 (socket says 172.21.15.155:56630) 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3622579611 0 0) 0x7faf3810a600 con 0x7faf380f9a30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf24003030 con 0x7faf380f9a30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3735680088 0 0) 0x7faf38115e60 con 0x7faf38101b30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf3810a600 con 0x7faf38101b30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777120155 0 0) 0x7faf38117040 con 0x7faf380fdf00 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf38115e60 con 0x7faf380fdf00 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 466201449 0 0) 0x7faf3810a600 con 0x7faf38101b30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf38117040 con 0x7faf38101b30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7faf30001bc0 con 0x7faf38101b30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 770769829 0 0) 0x7faf24003030 con 0x7faf380f9a30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf3810a600 con 0x7faf380f9a30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7faf20000fe0 con 0x7faf380f9a30 2024-02-18T01:17:22.367 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1062963827 0 0) 0x7faf38115e60 con 0x7faf380fdf00 2024-02-18T01:17:22.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7faf24003030 con 0x7faf380fdf00 2024-02-18T01:17:22.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2583991527 0 0) 0x7faf38117040 con 0x7faf38101b30 2024-02-18T01:17:22.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 >> v1:172.21.15.105:6790/0 conn(0x7faf380fdf00 legacy=0x7faf38110000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 >> v1:172.21.15.105:6789/0 conn(0x7faf380f9a30 legacy=0x7faf38109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf38118220 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf38117230 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7faf38117720 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7faf300026e0 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7faf30005e40 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.363+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf18004f20 con 0x7faf38101b30 2024-02-18T01:17:22.369 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.367+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 77081+0+0 (unknown 2048948923 0 0) 0x7faf30007050 con 0x7faf38101b30 2024-02-18T01:17:22.370 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.367+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 5884+0+0 (unknown 919150423 0 0) 0x7faf30078800 con 0x7faf38101b30 2024-02-18T01:17:22.379 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.375+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7faf30048960 con 0x7faf38101b30 2024-02-18T01:17:22.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:22 smithi105 bash[27669]: audit 2024-02-18T01:17:21.159751+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:22.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:22 smithi105 bash[27669]: audit 2024-02-18T01:17:21.162025+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:22.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:22 smithi105 bash[19734]: audit 2024-02-18T01:17:21.159751+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:22.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:22 smithi105 bash[19734]: audit 2024-02-18T01:17:21.162025+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:22.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[20805]: audit 2024-02-18T01:17:21.159751+0000 mon.a (mon.0) 587 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:22.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[20805]: audit 2024-02-18T01:17:21.162025+0000 mon.a (mon.0) 588 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:22.579 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.575+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7faf18000b90 con 0x7faf24062210 2024-02-18T01:17:22.595 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf2f7fe700 1 -- 172.21.15.155:0/3463448869 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (unknown 0 0 1342662408) 0x7faf18000b90 con 0x7faf24062210 2024-02-18T01:17:22.595 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled prometheus update... 2024-02-18T01:17:22.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 >> v1:172.21.15.105:6800/3087430119 conn(0x7faf24062210 legacy=0x7faf240646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:22.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 >> v1:172.21.15.155:6789/0 conn(0x7faf38101b30 legacy=0x7faf38113730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:22.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 shutdown_connections 2024-02-18T01:17:22.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 >> 172.21.15.155:0/3463448869 conn(0x7faf380f54b0 msgr2=0x7faf38103fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:22.597 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 shutdown_connections 2024-02-18T01:17:22.598 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:22.591+0000 7faf3cf22700 1 -- 172.21.15.155:0/3463448869 wait complete. 2024-02-18T01:17:22.966 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[43865]: log file path now is '/var/log/tcmu-runner.log' 2024-02-18T01:17:23.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[43865]: debug Started the configuration object watcher 2024-02-18T01:17:23.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[43865]: debug Checking for config object changes every 1s 2024-02-18T01:17:23.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:22 smithi155 bash[43865]: debug Processing osd blocklist entries for this node 2024-02-18T01:17:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[20805]: cluster 2024-02-18T01:17:21.561710+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v274: 132 pgs: 132 active+clean; 4.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.8 KiB/s wr, 209 op/s 2024-02-18T01:17:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[20805]: audit 2024-02-18T01:17:22.567972+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[20805]: audit 2024-02-18T01:17:22.593507+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.389 DEBUG:teuthology.orchestra.run.smithi155:prometheus.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@prometheus.a.service 2024-02-18T01:17:23.392 INFO:tasks.cephadm:Adding node-exporter.a on smithi105 2024-02-18T01:17:23.392 INFO:tasks.cephadm:Adding node-exporter.b on smithi155 2024-02-18T01:17:23.392 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply node-exporter '2;smithi105=a;smithi155=b' 2024-02-18T01:17:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[19734]: cluster 2024-02-18T01:17:21.561710+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v274: 132 pgs: 132 active+clean; 4.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.8 KiB/s wr, 209 op/s 2024-02-18T01:17:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[19734]: audit 2024-02-18T01:17:22.567972+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[19734]: audit 2024-02-18T01:17:22.593507+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[27669]: cluster 2024-02-18T01:17:21.561710+0000 mgr.y (mgr.14146) 284 : cluster [DBG] pgmap v274: 132 pgs: 132 active+clean; 4.4 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 97 KiB/s rd, 4.8 KiB/s wr, 209 op/s 2024-02-18T01:17:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[27669]: audit 2024-02-18T01:17:22.567972+0000 mon.a (mon.0) 589 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:23 smithi105 bash[27669]: audit 2024-02-18T01:17:22.593507+0000 mon.a (mon.0) 590 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:23.500 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:23.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[43865]: debug Reading the configuration object to update local LIO configuration 2024-02-18T01:17:23.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[43865]: debug Configuration does not have an entry for this host(smithi155.front.sepia.ceph.com) - nothing to define to LIO 2024-02-18T01:17:23.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:23 smithi155 bash[43865]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[27669]: audit 2024-02-18T01:17:22.580646+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24433 v1:172.21.15.155:0/3463448869' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[27669]: cephadm 2024-02-18T01:17:22.584680+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service prometheus spec with placement smithi155=a;count:1 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[27669]: audit 2024-02-18T01:17:23.275198+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.155:0/1068452142' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[19734]: audit 2024-02-18T01:17:22.580646+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24433 v1:172.21.15.155:0/3463448869' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[19734]: cephadm 2024-02-18T01:17:22.584680+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service prometheus spec with placement smithi155=a;count:1 2024-02-18T01:17:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:24 smithi105 bash[19734]: audit 2024-02-18T01:17:23.275198+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.155:0/1068452142' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T01:17:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:24 smithi155 bash[20805]: audit 2024-02-18T01:17:22.580646+0000 mgr.y (mgr.14146) 285 : audit [DBG] from='client.24433 v1:172.21.15.155:0/3463448869' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:24.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:24 smithi155 bash[20805]: cephadm 2024-02-18T01:17:22.584680+0000 mgr.y (mgr.14146) 286 : cephadm [INF] Saving service prometheus spec with placement smithi155=a;count:1 2024-02-18T01:17:24.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:24 smithi155 bash[20805]: audit 2024-02-18T01:17:23.275198+0000 mon.b (mon.1) 20 : audit [DBG] from='client.? v1:172.21.15.155:0/1068452142' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-02-18T01:17:25.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:25 smithi105 bash[27669]: cluster 2024-02-18T01:17:23.562677+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v275: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.8 KiB/s wr, 199 op/s 2024-02-18T01:17:25.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:25 smithi105 bash[19734]: cluster 2024-02-18T01:17:23.562677+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v275: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.8 KiB/s wr, 199 op/s 2024-02-18T01:17:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:25 smithi155 bash[20805]: cluster 2024-02-18T01:17:23.562677+0000 mgr.y (mgr.14146) 287 : cluster [DBG] pgmap v275: 132 pgs: 132 active+clean; 5.3 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 92 KiB/s rd, 4.8 KiB/s wr, 199 op/s 2024-02-18T01:17:26.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.467+0000 7f346f2af700 1 -- 172.21.15.155:0/4258586351 >> v1:172.21.15.155:6789/0 conn(0x7f34680fabf0 legacy=0x7f34680fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:26.473 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.467+0000 7f346f2af700 1 -- 172.21.15.155:0/4258586351 shutdown_connections 2024-02-18T01:17:26.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.467+0000 7f346f2af700 1 -- 172.21.15.155:0/4258586351 >> 172.21.15.155:0/4258586351 conn(0x7f34680f1e60 msgr2=0x7f34680f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:26.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.467+0000 7f346f2af700 1 -- 172.21.15.155:0/4258586351 shutdown_connections 2024-02-18T01:17:26.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.467+0000 7f346f2af700 1 -- 172.21.15.155:0/4258586351 wait complete. 2024-02-18T01:17:26.474 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 Processor -- start 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- start start 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3468112ae0 con 0x7f34680f6720 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3468113c80 con 0x7f34680fabf0 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f3468114e20 con 0x7f34680fe820 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346c84a700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f34680fabf0 0x7f346810dc90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:56720/0 (socket says 172.21.15.155:56720) 2024-02-18T01:17:26.475 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346c84a700 1 -- 172.21.15.155:0/1236757092 learned_addr learned my addr 172.21.15.155:0/1236757092 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3261737143 0 0) 0x7f3468113c80 con 0x7f34680fabf0 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3454003030 con 0x7f34680fabf0 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251824905 0 0) 0x7f3468112ae0 con 0x7f34680f6720 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3468113c80 con 0x7f34680f6720 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3708732819 0 0) 0x7f3468114e20 con 0x7f34680fe820 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3468112ae0 con 0x7f34680fe820 2024-02-18T01:17:26.476 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4031085944 0 0) 0x7f3454003030 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f3468114e20 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f3458001e30 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2597269949 0 0) 0x7f3468114e20 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 >> v1:172.21.15.105:6790/0 conn(0x7f34680fe820 legacy=0x7f34681113c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 >> v1:172.21.15.105:6789/0 conn(0x7f34680f6720 legacy=0x7f346810a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3468115fc0 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3468112cd0 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f3468113140 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f34580015b0 con 0x7f34680fabf0 2024-02-18T01:17:26.477 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f3458003b10 con 0x7f34680fabf0 2024-02-18T01:17:26.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.471+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f344c004f50 con 0x7f34680fabf0 2024-02-18T01:17:26.478 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.475+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 15) v1 ==== 77081+0+0 (unknown 2048948923 0 0) 0x7f34580032e0 con 0x7f34680fabf0 2024-02-18T01:17:26.479 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.475+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 5884+0+0 (unknown 919150423 0 0) 0x7f3458078b50 con 0x7f34680fabf0 2024-02-18T01:17:26.486 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.483+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f3458048b60 con 0x7f34680fabf0 2024-02-18T01:17:26.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:26 smithi105 bash[19734]: cluster 2024-02-18T01:17:25.563765+0000 mgr.y (mgr.14146) 288 : cluster [DBG] pgmap v276: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s rd, 4.2 KiB/s wr, 185 op/s 2024-02-18T01:17:26.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:26 smithi105 bash[27669]: cluster 2024-02-18T01:17:25.563765+0000 mgr.y (mgr.14146) 288 : cluster [DBG] pgmap v276: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s rd, 4.2 KiB/s wr, 185 op/s 2024-02-18T01:17:26.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:26 smithi155 bash[20805]: cluster 2024-02-18T01:17:25.563765+0000 mgr.y (mgr.14146) 288 : cluster [DBG] pgmap v276: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 87 KiB/s rd, 4.2 KiB/s wr, 185 op/s 2024-02-18T01:17:26.676 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.667+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi105=a;smithi155=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f344c000bc0 con 0x7f3454062160 2024-02-18T01:17:26.686 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f345e777700 1 -- 172.21.15.155:0/1236757092 <== mgr.14146 v1:172.21.15.105:6800/3087430119 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (unknown 0 0 240551134) 0x7f344c000bc0 con 0x7f3454062160 2024-02-18T01:17:26.687 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled node-exporter update... 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 >> v1:172.21.15.105:6800/3087430119 conn(0x7f3454062160 legacy=0x7f3454064620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 >> v1:172.21.15.155:6789/0 conn(0x7f34680fabf0 legacy=0x7f346810dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 shutdown_connections 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 >> 172.21.15.155:0/1236757092 conn(0x7f34680f1e60 msgr2=0x7f34680fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 shutdown_connections 2024-02-18T01:17:26.689 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:26.683+0000 7f346f2af700 1 -- 172.21.15.155:0/1236757092 wait complete. 2024-02-18T01:17:27.296 DEBUG:teuthology.orchestra.run.smithi105:node-exporter.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.a.service 2024-02-18T01:17:27.300 DEBUG:teuthology.orchestra.run.smithi155:node-exporter.b> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.b.service 2024-02-18T01:17:27.302 INFO:tasks.cephadm:Adding alertmanager.a on smithi105 2024-02-18T01:17:27.302 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply alertmanager '1;smithi105=a' 2024-02-18T01:17:27.363 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:27.500 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:27.692 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[19734]: audit 2024-02-18T01:17:26.674265+0000 mgr.y (mgr.14146) 289 : audit [DBG] from='client.24463 v1:172.21.15.155:0/1236757092' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi105=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:27.692 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[19734]: cephadm 2024-02-18T01:17:26.677832+0000 mgr.y (mgr.14146) 290 : cephadm [INF] Saving service node-exporter spec with placement smithi105=a;smithi155=b;count:2 2024-02-18T01:17:27.692 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[19734]: audit 2024-02-18T01:17:26.685017+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[27669]: audit 2024-02-18T01:17:26.674265+0000 mgr.y (mgr.14146) 289 : audit [DBG] from='client.24463 v1:172.21.15.155:0/1236757092' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi105=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[27669]: cephadm 2024-02-18T01:17:26.677832+0000 mgr.y (mgr.14146) 290 : cephadm [INF] Saving service node-exporter spec with placement smithi105=a;smithi155=b;count:2 2024-02-18T01:17:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:27 smithi105 bash[27669]: audit 2024-02-18T01:17:26.685017+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:27.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:27 smithi155 bash[20805]: audit 2024-02-18T01:17:26.674265+0000 mgr.y (mgr.14146) 289 : audit [DBG] from='client.24463 v1:172.21.15.155:0/1236757092' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi105=a;smithi155=b", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:27.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:27 smithi155 bash[20805]: cephadm 2024-02-18T01:17:26.677832+0000 mgr.y (mgr.14146) 290 : cephadm [INF] Saving service node-exporter spec with placement smithi105=a;smithi155=b;count:2 2024-02-18T01:17:27.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:27 smithi155 bash[20805]: audit 2024-02-18T01:17:26.685017+0000 mon.a (mon.0) 591 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:28.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:28 smithi105 bash[19734]: cluster 2024-02-18T01:17:27.564773+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 81 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-18T01:17:28.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:28 smithi105 bash[19734]: audit 2024-02-18T01:17:28.553487+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:28 smithi105 bash[27669]: cluster 2024-02-18T01:17:27.564773+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 81 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-18T01:17:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:28 smithi105 bash[27669]: audit 2024-02-18T01:17:28.553487+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:28 smithi155 bash[20805]: cluster 2024-02-18T01:17:27.564773+0000 mgr.y (mgr.14146) 291 : cluster [DBG] pgmap v277: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 81 KiB/s rd, 3.7 KiB/s wr, 170 op/s 2024-02-18T01:17:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:28 smithi155 bash[20805]: audit 2024-02-18T01:17:28.553487+0000 mon.a (mon.0) 592 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[19734]: audit 2024-02-18T01:17:28.837108+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[19734]: audit 2024-02-18T01:17:28.838372+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[19734]: audit 2024-02-18T01:17:28.845896+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[19734]: audit 2024-02-18T01:17:28.855491+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T01:17:29.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[20046]: ignoring --setuser ceph since I am not root 2024-02-18T01:17:29.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[20046]: ignoring --setgroup ceph since I am not root 2024-02-18T01:17:29.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[27669]: audit 2024-02-18T01:17:28.837108+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:29.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[27669]: audit 2024-02-18T01:17:28.838372+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:29.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[27669]: audit 2024-02-18T01:17:28.845896+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:29.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:29 smithi105 bash[27669]: audit 2024-02-18T01:17:28.855491+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[20805]: audit 2024-02-18T01:17:28.837108+0000 mon.a (mon.0) 593 : audit [DBG] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[20805]: audit 2024-02-18T01:17:28.838372+0000 mon.a (mon.0) 594 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[20805]: audit 2024-02-18T01:17:28.845896+0000 mon.a (mon.0) 595 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[20805]: audit 2024-02-18T01:17:28.855491+0000 mon.a (mon.0) 596 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[22101]: ignoring --setuser ceph since I am not root 2024-02-18T01:17:29.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:29 smithi155 bash[22101]: ignoring --setgroup ceph since I am not root 2024-02-18T01:17:30.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/3382038971 >> v1:172.21.15.155:6789/0 conn(0x7fb7d80fabf0 legacy=0x7fb7d80fcfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:30.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/3382038971 shutdown_connections 2024-02-18T01:17:30.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/3382038971 >> 172.21.15.155:0/3382038971 conn(0x7fb7d80f1e60 msgr2=0x7fb7d80f42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:30.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/3382038971 shutdown_connections 2024-02-18T01:17:30.355 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/3382038971 wait complete. 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 Processor -- start 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- start start 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7d8112d60 con 0x7fb7d80fabf0 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7d8113f00 con 0x7fb7d80fe820 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb7d81150a0 con 0x7fb7d80f6720 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d7fff700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7fb7d80fe820 0x7fb7d8111640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:56758/0 (socket says 172.21.15.155:56758) 2024-02-18T01:17:30.356 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d7fff700 1 -- 172.21.15.155:0/673818229 learned_addr learned my addr 172.21.15.155:0/673818229 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546484017 0 0) 0x7fb7d8113f00 con 0x7fb7d80fe820 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c4003030 con 0x7fb7d80fe820 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896641384 0 0) 0x7fb7d81150a0 con 0x7fb7d80f6720 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d8113f00 con 0x7fb7d80f6720 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 493203972 0 0) 0x7fb7d8112d60 con 0x7fb7d80fabf0 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7d81150a0 con 0x7fb7d80fabf0 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3925504539 0 0) 0x7fb7c4003030 con 0x7fb7d80fe820 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7d8112d60 con 0x7fb7d80fe820 2024-02-18T01:17:30.358 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb7d0001bb0 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2843827990 0 0) 0x7fb7d8113f00 con 0x7fb7d80f6720 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7c4003030 con 0x7fb7d80f6720 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb7c8001880 con 0x7fb7d80f6720 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3301761422 0 0) 0x7fb7d81150a0 con 0x7fb7d80fabf0 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb7d8113f00 con 0x7fb7d80fabf0 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2917615731 0 0) 0x7fb7d8112d60 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6790/0 conn(0x7fb7d80f6720 legacy=0x7fb7d810a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6789/0 conn(0x7fb7d80fabf0 legacy=0x7fb7d810df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7d8116240 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7d8115290 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.351+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7d8115780 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb7d0002760 con 0x7fb7d80fe820 2024-02-18T01:17:30.359 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb7d0005df0 con 0x7fb7d80fe820 2024-02-18T01:17:30.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7b8004f20 con 0x7fb7d80fe820 2024-02-18T01:17:30.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 16) v1 ==== 77095+0+0 (unknown 1188152848 0 0) 0x7fb7d00022c0 con 0x7fb7d80fe820 2024-02-18T01:17:30.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7d77fe700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3087430119 2024-02-18T01:17:30.360 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.355+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(61..61 src has 1..61) v4 ==== 5884+0+0 (unknown 919150423 0 0) 0x7fb7d0078880 con 0x7fb7d80fe820 2024-02-18T01:17:30.368 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.363+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb7d0048830 con 0x7fb7d80fe820 2024-02-18T01:17:30.489 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[20046]: debug 2024-02-18T01:17:30.059+0000 7f74ba886680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:17:30.490 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[20046]: debug 2024-02-18T01:17:30.159+0000 7f74ba886680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:17:30.494 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:30 smithi155 bash[22101]: debug 2024-02-18T01:17:30.059+0000 7fb97471b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-02-18T01:17:30.494 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:30 smithi155 bash[22101]: debug 2024-02-18T01:17:30.163+0000 7fb97471b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-02-18T01:17:30.562 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.559+0000 7fb7d77fe700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3087430119 2024-02-18T01:17:30.563 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.559+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6800/3087430119 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi105=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7b8000b90 con 0x7fb7c4062390 2024-02-18T01:17:30.856 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[20046]: debug 2024-02-18T01:17:30.591+0000 7f74ba886680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:17:30.966 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:30.963+0000 7fb7d77fe700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3087430119 2024-02-18T01:17:30.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:30 smithi155 bash[22101]: debug 2024-02-18T01:17:30.615+0000 7fb97471b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-02-18T01:17:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:30 smithi155 bash[20805]: audit 2024-02-18T01:17:29.863917+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T01:17:30.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:30 smithi155 bash[20805]: cluster 2024-02-18T01:17:29.864026+0000 mon.a (mon.0) 598 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-18T01:17:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[19734]: audit 2024-02-18T01:17:29.863917+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T01:17:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[19734]: cluster 2024-02-18T01:17:29.864026+0000 mon.a (mon.0) 598 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-18T01:17:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[27669]: audit 2024-02-18T01:17:29.863917+0000 mon.a (mon.0) 597 : audit [INF] from='mgr.14146 v1:172.21.15.105:0/2454331214' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-02-18T01:17:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:30 smithi105 bash[27669]: cluster 2024-02-18T01:17:29.864026+0000 mon.a (mon.0) 598 : cluster [DBG] mgrmap e16: y(active, since 7m), standbys: x 2024-02-18T01:17:31.770 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:31.767+0000 7fb7d77fe700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3087430119 2024-02-18T01:17:31.829 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:31 smithi105 bash[20046]: debug 2024-02-18T01:17:31.491+0000 7f74ba886680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:17:31.829 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:31 smithi105 bash[20046]: debug 2024-02-18T01:17:31.583+0000 7f74ba886680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:17:31.849 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:31 smithi155 bash[22101]: debug 2024-02-18T01:17:31.511+0000 7fb97471b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-02-18T01:17:31.850 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:31 smithi155 bash[22101]: debug 2024-02-18T01:17:31.603+0000 7fb97471b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-02-18T01:17:32.239 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:31 smithi105 bash[20046]: debug 2024-02-18T01:17:31.823+0000 7f74ba886680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:17:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:31 smithi155 bash[22101]: debug 2024-02-18T01:17:31.843+0000 7fb97471b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-02-18T01:17:33.188 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:32 smithi105 bash[20046]: debug 2024-02-18T01:17:32.927+0000 7f74ba886680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:17:33.188 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.087+0000 7f74ba886680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:17:33.188 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.183+0000 7f74ba886680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:17:33.221 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:32 smithi155 bash[22101]: debug 2024-02-18T01:17:32.955+0000 7fb97471b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-02-18T01:17:33.222 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:33 smithi155 bash[22101]: debug 2024-02-18T01:17:33.123+0000 7fb97471b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:17:33.374 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:33.371+0000 7fb7d77fe700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v1:172.21.15.105:6800/3087430119 2024-02-18T01:17:33.489 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.291+0000 7f74ba886680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:17:33.493 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:33 smithi155 bash[22101]: debug 2024-02-18T01:17:33.215+0000 7fb97471b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-02-18T01:17:33.494 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:33 smithi155 bash[22101]: debug 2024-02-18T01:17:33.327+0000 7fb97471b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-02-18T01:17:33.876 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.511+0000 7f74ba886680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:17:33.876 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.871+0000 7f74ba886680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:17:33.904 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:33 smithi155 bash[22101]: debug 2024-02-18T01:17:33.543+0000 7fb97471b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-02-18T01:17:34.240 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:33 smithi105 bash[20046]: debug 2024-02-18T01:17:33.983+0000 7f74ba886680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:17:34.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:33 smithi155 bash[22101]: debug 2024-02-18T01:17:33.899+0000 7fb97471b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-02-18T01:17:34.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:34 smithi155 bash[22101]: debug 2024-02-18T01:17:34.015+0000 7fb97471b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-02-18T01:17:35.239 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:34 smithi105 bash[20046]: debug 2024-02-18T01:17:34.839+0000 7f74ba886680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:17:35.240 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:34 smithi105 bash[20046]: debug 2024-02-18T01:17:34.947+0000 7f74ba886680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:17:35.240 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:35 smithi105 bash[20046]: debug 2024-02-18T01:17:35.063+0000 7f74ba886680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:17:35.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:34 smithi155 bash[22101]: debug 2024-02-18T01:17:34.859+0000 7fb97471b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-02-18T01:17:35.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:34 smithi155 bash[22101]: debug 2024-02-18T01:17:34.959+0000 7fb97471b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-02-18T01:17:35.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:35 smithi155 bash[22101]: debug 2024-02-18T01:17:35.071+0000 7fb97471b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-02-18T01:17:35.571 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:35 smithi105 bash[20046]: debug 2024-02-18T01:17:35.279+0000 7f74ba886680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:17:35.571 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:35 smithi105 bash[20046]: debug 2024-02-18T01:17:35.567+0000 7f74ba886680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:17:35.579 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:35 smithi155 bash[22101]: debug 2024-02-18T01:17:35.287+0000 7fb97471b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-02-18T01:17:35.898 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:35 smithi105 bash[20046]: debug 2024-02-18T01:17:35.727+0000 7f74ba886680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:17:35.903 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:35 smithi155 bash[22101]: debug 2024-02-18T01:17:35.575+0000 7fb97471b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-02-18T01:17:35.903 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:35 smithi155 bash[22101]: debug 2024-02-18T01:17:35.731+0000 7fb97471b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-02-18T01:17:36.215 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:35 smithi105 bash[20046]: debug 2024-02-18T01:17:35.895+0000 7f74ba886680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:17:36.215 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: debug 2024-02-18T01:17:36.103+0000 7f74ba886680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:17:36.215 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: debug 2024-02-18T01:17:36.211+0000 7f74ba886680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:17:36.217 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:35 smithi155 bash[22101]: debug 2024-02-18T01:17:35.899+0000 7fb97471b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-02-18T01:17:36.217 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: debug 2024-02-18T01:17:36.107+0000 7fb97471b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-02-18T01:17:36.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:36.247+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mgrmap(e 17) v1 ==== 76859+0+0 (unknown 731473364 0 0) 0x7fb7d0076540 con 0x7fb7d80fe820 2024-02-18T01:17:36.254 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:36.251+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/3087430119 conn(0x7fb7c4062390 legacy=0x7fb7c4064850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-02-18T01:17:36.479 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.237259+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T01:17:36.479 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.237514+0000 mon.a (mon.0) 600 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:17:36.479 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.238038+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-18T01:17:36.479 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.238324+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:17:36.479 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.238576+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.239167+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.250186+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.251141+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0127289s), standbys: x 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.254411+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.254884+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.256270+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.256468+0000 mon.a (mon.0) 606 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.256639+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.257034+0000 mon.a (mon.0) 608 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:17:36.480 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.257280+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.257503+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.257762+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.258001+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.258254+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.258505+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.258746+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.258980+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.259221+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.259476+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.259677+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: audit 2024-02-18T01:17:36.260149+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:17:36.481 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[19734]: cluster 2024-02-18T01:17:36.262853+0000 mon.a (mon.0) 621 : cluster [INF] Manager daemon y is now available 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.237259+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.237514+0000 mon.a (mon.0) 600 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.238038+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.238324+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.238576+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.239167+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.250186+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.251141+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0127289s), standbys: x 2024-02-18T01:17:36.482 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.254411+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.254884+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.256270+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.256468+0000 mon.a (mon.0) 606 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.256639+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.257034+0000 mon.a (mon.0) 608 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.257280+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.257503+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.257762+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.258001+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.258254+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.258505+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.258746+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.258980+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:17:36.483 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.259221+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.259476+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.259677+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: audit 2024-02-18T01:17:36.260149+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[27669]: cluster 2024-02-18T01:17:36.262853+0000 mon.a (mon.0) 621 : cluster [INF] Manager daemon y is now available 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: debug 2024-02-18T01:17:36.211+0000 7fb97471b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: [18/Feb/2024:01:17:36] ENGINE Bus STARTING 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: CherryPy Checker: 2024-02-18T01:17:36.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: The Application mounted at '' has an empty config. 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: [18/Feb/2024:01:17:36] ENGINE Serving on http://:::9283 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[22101]: [18/Feb/2024:01:17:36] ENGINE Bus STARTED 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.237259+0000 mon.a (mon.0) 599 : cluster [DBG] Standby manager daemon x restarted 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.237514+0000 mon.a (mon.0) 600 : cluster [DBG] Standby manager daemon x started 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.238038+0000 mon.a (mon.0) 601 : cluster [INF] Active manager daemon y restarted 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.238324+0000 mon.b (mon.1) 21 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.238576+0000 mon.a (mon.0) 602 : cluster [INF] Activating manager daemon y 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.239167+0000 mon.b (mon.1) 22 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.250186+0000 mon.a (mon.0) 603 : cluster [DBG] osdmap e62: 8 total, 8 up, 8 in 2024-02-18T01:17:36.485 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.251141+0000 mon.a (mon.0) 604 : cluster [DBG] mgrmap e17: y(active, starting, since 0.0127289s), standbys: x 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.254411+0000 mon.b (mon.1) 23 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.254884+0000 mon.b (mon.1) 24 : audit [DBG] from='mgr.? v1:172.21.15.155:0/2075802013' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.256270+0000 mon.a (mon.0) 605 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.256468+0000 mon.a (mon.0) 606 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.256639+0000 mon.a (mon.0) 607 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.257034+0000 mon.a (mon.0) 608 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.257280+0000 mon.a (mon.0) 609 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.257503+0000 mon.a (mon.0) 610 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.257762+0000 mon.a (mon.0) 611 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.258001+0000 mon.a (mon.0) 612 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.258254+0000 mon.a (mon.0) 613 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.258505+0000 mon.a (mon.0) 614 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.258746+0000 mon.a (mon.0) 615 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-02-18T01:17:36.486 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.258980+0000 mon.a (mon.0) 616 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-02-18T01:17:36.487 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.259221+0000 mon.a (mon.0) 617 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-02-18T01:17:36.487 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.259476+0000 mon.a (mon.0) 618 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-02-18T01:17:36.487 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.259677+0000 mon.a (mon.0) 619 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-02-18T01:17:36.487 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: audit 2024-02-18T01:17:36.260149+0000 mon.a (mon.0) 620 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-02-18T01:17:36.487 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:36 smithi155 bash[20805]: cluster 2024-02-18T01:17:36.262853+0000 mon.a (mon.0) 621 : cluster [INF] Manager daemon y is now available 2024-02-18T01:17:36.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: Warning: Permanently added '172.21.15.105' (ECDSA) to the list of known hosts. 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: [18/Feb/2024:01:17:36] ENGINE Bus STARTING 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: Warning: Permanently added '172.21.15.155' (ECDSA) to the list of known hosts. 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: CherryPy Checker: 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: The Application mounted at '' has an empty config. 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: [18/Feb/2024:01:17:36] ENGINE Serving on http://:::9283 2024-02-18T01:17:36.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:17:36 smithi105 bash[20046]: [18/Feb/2024:01:17:36] ENGINE Bus STARTED 2024-02-18T01:17:37.260 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.255+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mon.1 v1:172.21.15.155:6789/0 11 ==== mgrmap(e 18) v1 ==== 76894+0+0 (unknown 225577743 0 0) 0x7fb7d00768c0 con 0x7fb7d80fe820 2024-02-18T01:17:37.261 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.255+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 --> v1:172.21.15.105:6800/2148176012 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi105=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fb7b8000b90 con 0x7fb7c406c610 2024-02-18T01:17:37.274 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7d57fa700 1 -- 172.21.15.155:0/673818229 <== mgr.24475 v1:172.21.15.105:6800/2148176012 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (unknown 0 0 1850065467) 0x7fb7b8000b90 con 0x7fb7c406c610 2024-02-18T01:17:37.274 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled alertmanager update... 2024-02-18T01:17:37.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.105:6800/2148176012 conn(0x7fb7c406c610 legacy=0x7fb7c406ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:37.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 >> v1:172.21.15.155:6789/0 conn(0x7fb7d80fe820 legacy=0x7fb7d8111640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:37.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7d7fff700 1 -- 172.21.15.155:0/673818229 reap_dead start 2024-02-18T01:17:37.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 shutdown_connections 2024-02-18T01:17:37.277 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 >> 172.21.15.155:0/673818229 conn(0x7fb7d80f1e60 msgr2=0x7fb7d80fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 shutdown_connections 2024-02-18T01:17:37.278 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:37.271+0000 7fb7de04a700 1 -- 172.21.15.155:0/673818229 wait complete. 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:36.405368+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:36.407951+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:36.473772+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:36.492358+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:36.528034+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:17:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: cluster 2024-02-18T01:17:37.261764+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e18: y(active, since 1.02335s), standbys: x 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[19734]: audit 2024-02-18T01:17:37.272806+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:36.405368+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:36.407951+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:36.473772+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:36.492358+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:36.528034+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:17:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: cluster 2024-02-18T01:17:37.261764+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e18: y(active, since 1.02335s), standbys: x 2024-02-18T01:17:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:37 smithi105 bash[27669]: audit 2024-02-18T01:17:37.272806+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:37.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:36.405368+0000 mon.a (mon.0) 622 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:36.407951+0000 mon.a (mon.0) 623 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:36.473772+0000 mon.a (mon.0) 624 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:36.492358+0000 mon.a (mon.0) 625 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:36.528034+0000 mon.a (mon.0) 626 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: cluster 2024-02-18T01:17:37.261764+0000 mon.a (mon.0) 627 : cluster [DBG] mgrmap e18: y(active, since 1.02335s), standbys: x 2024-02-18T01:17:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:37 smithi155 bash[20805]: audit 2024-02-18T01:17:37.272806+0000 mon.a (mon.0) 628 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:38.002 DEBUG:teuthology.orchestra.run.smithi105:alertmanager.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@alertmanager.a.service 2024-02-18T01:17:38.005 INFO:tasks.cephadm:Adding grafana.a on smithi155 2024-02-18T01:17:38.005 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph orch apply grafana '1;smithi155=a' 2024-02-18T01:17:38.141 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:38 smithi105 bash[19734]: cephadm 2024-02-18T01:17:37.268390+0000 mgr.y (mgr.24475) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi105=a;count:1 2024-02-18T01:17:38.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:38 smithi105 bash[27669]: cephadm 2024-02-18T01:17:37.268390+0000 mgr.y (mgr.24475) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi105=a;count:1 2024-02-18T01:17:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:38 smithi155 bash[20805]: cephadm 2024-02-18T01:17:37.268390+0000 mgr.y (mgr.24475) 4 : cephadm [INF] Saving service alertmanager spec with placement smithi105=a;count:1 2024-02-18T01:17:39.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:39 smithi155 bash[20805]: cluster 2024-02-18T01:17:38.258591+0000 mgr.y (mgr.24475) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:39.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:39 smithi155 bash[20805]: cluster 2024-02-18T01:17:39.264855+0000 mon.a (mon.0) 629 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-18T01:17:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:39 smithi105 bash[19734]: cluster 2024-02-18T01:17:38.258591+0000 mgr.y (mgr.24475) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:39 smithi105 bash[19734]: cluster 2024-02-18T01:17:39.264855+0000 mon.a (mon.0) 629 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-18T01:17:39.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:39 smithi105 bash[27669]: cluster 2024-02-18T01:17:38.258591+0000 mgr.y (mgr.24475) 5 : cluster [DBG] pgmap v4: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:39.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:39 smithi105 bash[27669]: cluster 2024-02-18T01:17:39.264855+0000 mon.a (mon.0) 629 : cluster [DBG] mgrmap e19: y(active, since 3s), standbys: x 2024-02-18T01:17:40.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- 172.21.15.155:0/201365721 >> v1:172.21.15.155:6789/0 conn(0x7f45f8101b10 legacy=0x7f45f8103fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:40.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- 172.21.15.155:0/201365721 shutdown_connections 2024-02-18T01:17:40.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- 172.21.15.155:0/201365721 >> 172.21.15.155:0/201365721 conn(0x7f45f80f1e50 msgr2=0x7f45f80f4290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:40.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- 172.21.15.155:0/201365721 shutdown_connections 2024-02-18T01:17:40.972 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- 172.21.15.155:0/201365721 wait complete. 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 Processor -- start 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- start start 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45f810a540 con 0x7f45f8101b10 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45f8115da0 con 0x7f45f80fdf10 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45ff3a5700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f45f8116f80 con 0x7f45f80616e0 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45fc940700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f45f80fdf10 0x7f45f810ff40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:50300/0 (socket says 172.21.15.155:50300) 2024-02-18T01:17:40.973 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.967+0000 7f45fc940700 1 -- 172.21.15.155:0/3687688040 learned_addr learned my addr 172.21.15.155:0/3687688040 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:40.974 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1977850835 0 0) 0x7f45f8115da0 con 0x7f45f80fdf10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45e40033f0 con 0x7f45f80fdf10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 772124391 0 0) 0x7f45f8116f80 con 0x7f45f80616e0 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f8115da0 con 0x7f45f80616e0 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1120459811 0 0) 0x7f45f810a540 con 0x7f45f8101b10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45f8116f80 con 0x7f45f8101b10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 56034450 0 0) 0x7f45e40033f0 con 0x7f45f80fdf10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45f810a540 con 0x7f45f80fdf10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f45e80029d0 con 0x7f45f80fdf10 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3317209859 0 0) 0x7f45f8115da0 con 0x7f45f80616e0 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45e40033f0 con 0x7f45f80616e0 2024-02-18T01:17:40.975 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f45f40018c0 con 0x7f45f80616e0 2024-02-18T01:17:40.976 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4105359565 0 0) 0x7f45f8116f80 con 0x7f45f8101b10 2024-02-18T01:17:40.976 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f45f8115da0 con 0x7f45f8101b10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f45f00030a0 con 0x7f45f8101b10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2875071259 0 0) 0x7f45f810a540 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 >> v1:172.21.15.105:6790/0 conn(0x7f45f80616e0 legacy=0x7f45f8109970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 >> v1:172.21.15.105:6789/0 conn(0x7f45f8101b10 legacy=0x7f45f8113670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f8118160 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45f8117170 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f45f8117660 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f45e8002da0 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f45e8005b70 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45dc004f20 con 0x7f45f80fdf10 2024-02-18T01:17:40.977 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.971+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f45e8001ce0 con 0x7f45f80fdf10 2024-02-18T01:17:40.978 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.975+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7f45e80785f0 con 0x7f45f80fdf10 2024-02-18T01:17:40.984 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:40.979+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f45e804c220 con 0x7f45f80fdf10 2024-02-18T01:17:41.188 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.183+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 --> v1:172.21.15.105:6800/2148176012 -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f45dc000b90 con 0x7f45e40623d0 2024-02-18T01:17:41.202 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ee7fc700 1 -- 172.21.15.155:0/3687688040 <== mgr.24475 v1:172.21.15.105:6800/2148176012 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (unknown 0 0 664801700) 0x7f45dc000b90 con 0x7f45e40623d0 2024-02-18T01:17:41.202 INFO:teuthology.orchestra.run.smithi155.stdout:Scheduled grafana update... 2024-02-18T01:17:41.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 >> v1:172.21.15.105:6800/2148176012 conn(0x7f45e40623d0 legacy=0x7f45e4064890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:41.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 >> v1:172.21.15.155:6789/0 conn(0x7f45f80fdf10 legacy=0x7f45f810ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:41.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 shutdown_connections 2024-02-18T01:17:41.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 >> 172.21.15.155:0/3687688040 conn(0x7f45f80f1e50 msgr2=0x7f45f80f2230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:41.208 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 shutdown_connections 2024-02-18T01:17:41.209 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:41.199+0000 7f45ff3a5700 1 -- 172.21.15.155:0/3687688040 wait complete. 2024-02-18T01:17:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:41 smithi105 bash[19734]: cluster 2024-02-18T01:17:40.259313+0000 mgr.y (mgr.24475) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:41 smithi105 bash[19734]: audit 2024-02-18T01:17:41.200445+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:41.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:41 smithi105 bash[27669]: cluster 2024-02-18T01:17:40.259313+0000 mgr.y (mgr.24475) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:41.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:41 smithi105 bash[27669]: audit 2024-02-18T01:17:41.200445+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:41.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:41 smithi155 bash[20805]: cluster 2024-02-18T01:17:40.259313+0000 mgr.y (mgr.24475) 6 : cluster [DBG] pgmap v5: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:41.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:41 smithi155 bash[20805]: audit 2024-02-18T01:17:41.200445+0000 mon.a (mon.0) 630 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:41.883 DEBUG:teuthology.orchestra.run.smithi155:grafana.a> sudo journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@grafana.a.service 2024-02-18T01:17:41.886 INFO:tasks.cephadm:Setting up client nodes... 2024-02-18T01:17:41.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-18T01:17:41.997 INFO:journalctl@ceph.grafana.a.smithi155.stdout:-- Logs begin at Sun 2022-10-16 12:17:23 UTC. -- 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[19734]: audit 2024-02-18T01:17:41.189755+0000 mgr.y (mgr.24475) 7 : audit [DBG] from='client.24499 v1:172.21.15.155:0/3687688040' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[19734]: cephadm 2024-02-18T01:17:41.193033+0000 mgr.y (mgr.24475) 8 : cephadm [INF] Saving service grafana spec with placement smithi155=a;count:1 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[19734]: audit 2024-02-18T01:17:42.219585+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[27669]: audit 2024-02-18T01:17:41.189755+0000 mgr.y (mgr.24475) 7 : audit [DBG] from='client.24499 v1:172.21.15.155:0/3687688040' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[27669]: cephadm 2024-02-18T01:17:41.193033+0000 mgr.y (mgr.24475) 8 : cephadm [INF] Saving service grafana spec with placement smithi155=a;count:1 2024-02-18T01:17:42.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:42 smithi105 bash[27669]: audit 2024-02-18T01:17:42.219585+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:42.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:42 smithi155 bash[20805]: audit 2024-02-18T01:17:41.189755+0000 mgr.y (mgr.24475) 7 : audit [DBG] from='client.24499 v1:172.21.15.155:0/3687688040' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi155=a", "target": ["mon-mgr", ""]}]: dispatch 2024-02-18T01:17:42.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:42 smithi155 bash[20805]: cephadm 2024-02-18T01:17:41.193033+0000 mgr.y (mgr.24475) 8 : cephadm [INF] Saving service grafana spec with placement smithi155=a;count:1 2024-02-18T01:17:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:42 smithi155 bash[20805]: audit 2024-02-18T01:17:42.219585+0000 mon.a (mon.0) 631 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: cluster 2024-02-18T01:17:42.259864+0000 mgr.y (mgr.24475) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:42.499141+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:42.500741+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:42.502205+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:42.503626+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: cephadm 2024-02-18T01:17:42.504810+0000 mgr.y (mgr.24475) 10 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3817M 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:42.511458+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.558 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[19734]: audit 2024-02-18T01:17:43.133243+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: cluster 2024-02-18T01:17:42.259864+0000 mgr.y (mgr.24475) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:42.499141+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:42.500741+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:42.502205+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:42.503626+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: cephadm 2024-02-18T01:17:42.504810+0000 mgr.y (mgr.24475) 10 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3817M 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:42.511458+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.559 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:43 smithi105 bash[27669]: audit 2024-02-18T01:17:43.133243+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: cluster 2024-02-18T01:17:42.259864+0000 mgr.y (mgr.24475) 9 : cluster [DBG] pgmap v6: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:42.499141+0000 mon.a (mon.0) 632 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:42.500741+0000 mon.a (mon.0) 633 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:42.502205+0000 mon.a (mon.0) 634 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:42.503626+0000 mon.a (mon.0) 635 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: cephadm 2024-02-18T01:17:42.504810+0000 mgr.y (mgr.24475) 10 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3817M 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:42.511458+0000 mon.a (mon.0) 636 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:43 smithi155 bash[20805]: audit 2024-02-18T01:17:43.133243+0000 mon.a (mon.0) 637 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.239 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:44 smithi105 systemd[1]: Started Ceph node-exporter.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:17:44.240 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[51204]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-18T01:17:44.642 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.428199+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.430368+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.432271+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.433958+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.434999+0000 mgr.y (mgr.24475) 11 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4073M 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.443440+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.445011+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.446483+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.448531+0000 mgr.y (mgr.24475) 12 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.448898+0000 mgr.y (mgr.24475) 13 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.474018+0000 mgr.y (mgr.24475) 14 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.495361+0000 mgr.y (mgr.24475) 15 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.521298+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.528436+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.643 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:43.537092+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: cephadm 2024-02-18T01:17:43.541751+0000 mgr.y (mgr.24475) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi105 2024-02-18T01:17:44.644 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[20805]: audit 2024-02-18T01:17:44.087185+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.644 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:44 smithi155 systemd[1]: Started Ceph node-exporter.b for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.428199+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.430368+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.432271+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.433958+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.434999+0000 mgr.y (mgr.24475) 11 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4073M 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.443440+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.445011+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.446483+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.448531+0000 mgr.y (mgr.24475) 12 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.448898+0000 mgr.y (mgr.24475) 13 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.474018+0000 mgr.y (mgr.24475) 14 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.495361+0000 mgr.y (mgr.24475) 15 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.521298+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.528436+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:43.537092+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: cephadm 2024-02-18T01:17:43.541751+0000 mgr.y (mgr.24475) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi105 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[19734]: audit 2024-02-18T01:17:44.087185+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.428199+0000 mon.a (mon.0) 638 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.430368+0000 mon.a (mon.0) 639 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.432271+0000 mon.a (mon.0) 640 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.433958+0000 mon.a (mon.0) 641 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.434999+0000 mgr.y (mgr.24475) 11 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 4073M 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.443440+0000 mon.a (mon.0) 642 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.445011+0000 mon.a (mon.0) 643 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.446483+0000 mon.a (mon.0) 644 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.448531+0000 mgr.y (mgr.24475) 12 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.conf 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.448898+0000 mgr.y (mgr.24475) 13 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.conf 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.474018+0000 mgr.y (mgr.24475) 14 : cephadm [INF] Updating smithi105:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.495361+0000 mgr.y (mgr.24475) 15 : cephadm [INF] Updating smithi155:/etc/ceph/ceph.client.admin.keyring 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.521298+0000 mon.a (mon.0) 645 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.528436+0000 mon.a (mon.0) 646 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:43.537092+0000 mon.a (mon.0) 647 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: cephadm 2024-02-18T01:17:43.541751+0000 mgr.y (mgr.24475) 16 : cephadm [INF] Deploying daemon node-exporter.a on smithi105 2024-02-18T01:17:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:44 smithi105 bash[27669]: audit 2024-02-18T01:17:44.087185+0000 mon.a (mon.0) 648 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:44.994 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:44 smithi155 bash[45658]: Unable to find image 'quay.io/prometheus/node-exporter:v1.3.1' locally 2024-02-18T01:17:45.469 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-18T01:17:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.639+0000 7f409fd7c700 1 -- 172.21.15.105:0/965169992 >> v1:172.21.15.105:6789/0 conn(0x7f40980ff580 legacy=0x7f4098101970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.639+0000 7f409fd7c700 1 -- 172.21.15.105:0/965169992 shutdown_connections 2024-02-18T01:17:45.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.639+0000 7f409fd7c700 1 -- 172.21.15.105:0/965169992 >> 172.21.15.105:0/965169992 conn(0x7f409806c760 msgr2=0x7f409806cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.639+0000 7f409fd7c700 1 -- 172.21.15.105:0/965169992 shutdown_connections 2024-02-18T01:17:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.639+0000 7f409fd7c700 1 -- 172.21.15.105:0/965169992 wait complete. 2024-02-18T01:17:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 Processor -- start 2024-02-18T01:17:45.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- start start 2024-02-18T01:17:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4098116150 con 0x7f40981030c0 2024-02-18T01:17:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4098117330 con 0x7f40980fc780 2024-02-18T01:17:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4098118530 con 0x7f40980ff580 2024-02-18T01:17:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409d317700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f40980ff580 0x7f40980f9070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:56626/0 (socket says 172.21.15.105:56626) 2024-02-18T01:17:45.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409d317700 1 -- 172.21.15.105:0/600129143 learned_addr learned my addr 172.21.15.105:0/600129143 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154333621 0 0) 0x7f4098118530 con 0x7f40980ff580 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40840033f0 con 0x7f40980ff580 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1096339144 0 0) 0x7f4098117330 con 0x7f40980fc780 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4098118530 con 0x7f40980fc780 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551212960 0 0) 0x7f4098116150 con 0x7f40981030c0 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4098117330 con 0x7f40981030c0 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2972414625 0 0) 0x7f4098118530 con 0x7f40980fc780 2024-02-18T01:17:45.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4098116150 con 0x7f40980fc780 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 293310419 0 0) 0x7f40840033f0 con 0x7f40980ff580 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4098118530 con 0x7f40980ff580 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f40940018c0 con 0x7f40980fc780 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4088002c60 con 0x7f40980ff580 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2996813072 0 0) 0x7f4098116150 con 0x7f40980fc780 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 >> v1:172.21.15.105:6790/0 conn(0x7f40980ff580 legacy=0x7f40980f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 >> v1:172.21.15.105:6789/0 conn(0x7f40981030c0 legacy=0x7f4098114810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:45.649 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098119730 con 0x7f40980fc780 2024-02-18T01:17:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4098117520 con 0x7f40980fc780 2024-02-18T01:17:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4098117990 con 0x7f40980fc780 2024-02-18T01:17:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4094001f80 con 0x7f40980fc780 2024-02-18T01:17:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.643+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4094003230 con 0x7f40980fc780 2024-02-18T01:17:45.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.647+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f407c004f20 con 0x7f40980fc780 2024-02-18T01:17:45.651 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.647+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f4094001ae0 con 0x7f40980fc780 2024-02-18T01:17:45.652 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.647+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7f4094075c80 con 0x7f40980fc780 2024-02-18T01:17:45.657 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.651+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4094049850 con 0x7f40980fc780 2024-02-18T01:17:45.685 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: v1.3.1: Pulling from prometheus/node-exporter 2024-02-18T01:17:45.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[19734]: cephadm 2024-02-18T01:17:44.088657+0000 mgr.y (mgr.24475) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi155 2024-02-18T01:17:45.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[19734]: cluster 2024-02-18T01:17:44.261308+0000 mgr.y (mgr.24475) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[19734]: audit 2024-02-18T01:17:44.684378+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[19734]: cephadm 2024-02-18T01:17:44.714679+0000 mgr.y (mgr.24475) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi155 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[27669]: cephadm 2024-02-18T01:17:44.088657+0000 mgr.y (mgr.24475) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi155 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[27669]: cluster 2024-02-18T01:17:44.261308+0000 mgr.y (mgr.24475) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[27669]: audit 2024-02-18T01:17:44.684378+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[27669]: cephadm 2024-02-18T01:17:44.714679+0000 mgr.y (mgr.24475) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi155 2024-02-18T01:17:45.740 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: aa2a8d90b84c: Pulling fs layer 2024-02-18T01:17:45.741 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: b45d31ee2d7f: Pulling fs layer 2024-02-18T01:17:45.741 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: b5db1e299295: Pulling fs layer 2024-02-18T01:17:45.741 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: b45d31ee2d7f: Download complete 2024-02-18T01:17:45.741 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: aa2a8d90b84c: Verifying Checksum 2024-02-18T01:17:45.741 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: aa2a8d90b84c: Download complete 2024-02-18T01:17:45.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.927+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f407c005750 con 0x7f40980fc780 2024-02-18T01:17:45.938 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f408effd700 1 -- 172.21.15.105:0/600129143 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (unknown 57899324 0 3019756043) 0x7f4094047c60 con 0x7f40980fc780 2024-02-18T01:17:45.939 INFO:teuthology.orchestra.run.smithi105.stdout:[client.0] 2024-02-18T01:17:45.939 INFO:teuthology.orchestra.run.smithi105.stdout: key = AQC5WtFl+xt7NxAA/WiOyzlY7Yhv/a71W5W1nw== 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 >> v1:172.21.15.105:6800/2148176012 conn(0x7f4084062210 legacy=0x7f40840646d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 >> v1:172.21.15.155:6789/0 conn(0x7f40980fc780 legacy=0x7f40980f8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 shutdown_connections 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 >> 172.21.15.105:0/600129143 conn(0x7f409806c760 msgr2=0x7f409806d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 shutdown_connections 2024-02-18T01:17:45.941 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:17:45.935+0000 7f409fd7c700 1 -- 172.21.15.105:0/600129143 wait complete. 2024-02-18T01:17:45.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: aa2a8d90b84c: Pulling fs layer 2024-02-18T01:17:45.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: b45d31ee2d7f: Pulling fs layer 2024-02-18T01:17:45.958 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: b5db1e299295: Pulling fs layer 2024-02-18T01:17:45.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[20805]: cephadm 2024-02-18T01:17:44.088657+0000 mgr.y (mgr.24475) 17 : cephadm [INF] Deploying daemon node-exporter.b on smithi155 2024-02-18T01:17:45.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[20805]: cluster 2024-02-18T01:17:44.261308+0000 mgr.y (mgr.24475) 18 : cluster [DBG] pgmap v7: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 732 B/s rd, 0 op/s 2024-02-18T01:17:45.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[20805]: audit 2024-02-18T01:17:44.684378+0000 mon.a (mon.0) 649 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:45.959 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[20805]: cephadm 2024-02-18T01:17:44.714679+0000 mgr.y (mgr.24475) 19 : cephadm [INF] Deploying daemon prometheus.a on smithi155 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: b45d31ee2d7f: Verifying Checksum 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: b45d31ee2d7f: Download complete 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: aa2a8d90b84c: Verifying Checksum 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:45 smithi155 bash[45658]: aa2a8d90b84c: Download complete 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[45658]: b5db1e299295: Verifying Checksum 2024-02-18T01:17:46.212 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[45658]: b5db1e299295: Download complete 2024-02-18T01:17:46.239 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: b5db1e299295: Verifying Checksum 2024-02-18T01:17:46.239 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: b5db1e299295: Download complete 2024-02-18T01:17:46.240 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:45 smithi105 bash[51204]: aa2a8d90b84c: Pull complete 2024-02-18T01:17:46.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[45658]: aa2a8d90b84c: Pull complete 2024-02-18T01:17:46.989 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[51204]: b45d31ee2d7f: Pull complete 2024-02-18T01:17:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[27669]: audit 2024-02-18T01:17:45.929905+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.105:0/600129143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[27669]: audit 2024-02-18T01:17:45.930475+0000 mon.a (mon.0) 650 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[27669]: audit 2024-02-18T01:17:45.935012+0000 mon.a (mon.0) 651 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[27669]: audit 2024-02-18T01:17:46.482705+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[19734]: audit 2024-02-18T01:17:45.929905+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.105:0/600129143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[19734]: audit 2024-02-18T01:17:45.930475+0000 mon.a (mon.0) 650 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[19734]: audit 2024-02-18T01:17:45.935012+0000 mon.a (mon.0) 651 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:46 smithi105 bash[19734]: audit 2024-02-18T01:17:46.482705+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:46.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[20805]: audit 2024-02-18T01:17:45.929905+0000 mon.b (mon.1) 25 : audit [INF] from='client.? v1:172.21.15.105:0/600129143' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[20805]: audit 2024-02-18T01:17:45.930475+0000 mon.a (mon.0) 650 : audit [INF] from='client.24505 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:46.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[20805]: audit 2024-02-18T01:17:45.935012+0000 mon.a (mon.0) 651 : audit [INF] from='client.24505 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:46.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[20805]: audit 2024-02-18T01:17:46.482705+0000 mon.a (mon.0) 652 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:17:46.995 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:46 smithi155 bash[45658]: b45d31ee2d7f: Pull complete 2024-02-18T01:17:47.001 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:17:47.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-02-18T01:17:47.001 DEBUG:teuthology.orchestra.run.smithi105:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-02-18T01:17:47.019 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-02-18T01:17:47.690 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:47 smithi105 bash[51204]: b5db1e299295: Pull complete 2024-02-18T01:17:47.690 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:47 smithi105 bash[51204]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-18T01:17:47.690 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:47 smithi105 bash[51204]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-18T01:17:47.712 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:17:47.713 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:47 smithi155 bash[45658]: b5db1e299295: Pull complete 2024-02-18T01:17:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:47 smithi105 bash[19734]: cluster 2024-02-18T01:17:46.261994+0000 mgr.y (mgr.24475) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-18T01:17:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:47 smithi105 bash[27669]: cluster 2024-02-18T01:17:46.261994+0000 mgr.y (mgr.24475) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-18T01:17:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:47 smithi155 bash[20805]: cluster 2024-02-18T01:17:46.261994+0000 mgr.y (mgr.24475) 20 : cluster [DBG] pgmap v8: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 569 B/s rd, 0 op/s 2024-02-18T01:17:48.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:48 smithi105 bash[19734]: audit 2024-02-18T01:17:47.269953+0000 mgr.y (mgr.24475) 21 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:48 smithi105 bash[27669]: audit 2024-02-18T01:17:47.269953+0000 mgr.y (mgr.24475) 21 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:48 smithi155 bash[20805]: audit 2024-02-18T01:17:47.269953+0000 mgr.y (mgr.24475) 21 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:48.994 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:48 smithi155 bash[45658]: Digest: sha256:f2269e73124dd0f60a7d19a2ce1264d33d08a985aed0ee6b0b89d0be470592cd 2024-02-18T01:17:49.490 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.018Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-18T01:17:49.490 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.018Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-18T01:17:49.490 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.019Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.019Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.019Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.019Z caller=node_exporter.go:115 level=info collector=arp 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.019Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=edac 2024-02-18T01:17:49.491 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-18T01:17:49.492 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=os 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=stat 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=time 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-18T01:17:49.493 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=uname 2024-02-18T01:17:49.494 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-18T01:17:49.494 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-18T01:17:49.494 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-18T01:17:49.494 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-18T01:17:49.494 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[51204]: ts=2024-02-18T01:17:49.020Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-18T01:17:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[19734]: cluster 2024-02-18T01:17:48.263473+0000 mgr.y (mgr.24475) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-18T01:17:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:49 smithi105 bash[27669]: cluster 2024-02-18T01:17:48.263473+0000 mgr.y (mgr.24475) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-18T01:17:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:49 smithi155 bash[20805]: cluster 2024-02-18T01:17:48.263473+0000 mgr.y (mgr.24475) 22 : cluster [DBG] pgmap v9: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 931 B/s rd, 0 op/s 2024-02-18T01:17:49.994 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:49 smithi155 bash[45658]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.3.1 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.354Z caller=node_exporter.go:182 level=info msg="Starting node_exporter" version="(version=1.3.1, branch=HEAD, revision=a2321e7b940ddcff26873612bccdf7cd4c42b6b6)" 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.354Z caller=node_exporter.go:183 level=info msg="Build context" build_context="(go=go1.17.3, user=root@243aafa5525c, date=20211205-11:09:49)" 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+)($|/) 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:108 level=info msg="Enabled collectors" 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=arp 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=bcache 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=bonding 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=btrfs 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=conntrack 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=cpu 2024-02-18T01:17:51.495 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=cpufreq 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=diskstats 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=dmi 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=edac 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=entropy 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=fibrechannel 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=filefd 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=filesystem 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=hwmon 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=infiniband 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=ipvs 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=loadavg 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=mdadm 2024-02-18T01:17:51.496 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=meminfo 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=netclass 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=netdev 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=netstat 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=nfs 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=nfsd 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=nvme 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=os 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=powersupplyclass 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=pressure 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=rapl 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=schedstat 2024-02-18T01:17:51.497 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=sockstat 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=softnet 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=stat 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=tapestats 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.356Z caller=node_exporter.go:115 level=info collector=textfile 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=thermal_zone 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=time 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=udp_queues 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=uname 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=vmstat 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=xfs 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:115 level=info collector=zfs 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=node_exporter.go:199 level=info msg="Listening on" address=:9100 2024-02-18T01:17:51.498 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[45658]: ts=2024-02-18T01:17:51.357Z caller=tls_config.go:195 level=info msg="TLS is disabled." http2=false 2024-02-18T01:17:51.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:51 smithi105 bash[27669]: cluster 2024-02-18T01:17:50.264221+0000 mgr.y (mgr.24475) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:51 smithi105 bash[19734]: cluster 2024-02-18T01:17:50.264221+0000 mgr.y (mgr.24475) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:51.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:51 smithi155 bash[20805]: cluster 2024-02-18T01:17:50.264221+0000 mgr.y (mgr.24475) 23 : cluster [DBG] pgmap v10: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:53 smithi105 bash[19734]: cluster 2024-02-18T01:17:52.265362+0000 mgr.y (mgr.24475) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:53 smithi105 bash[27669]: cluster 2024-02-18T01:17:52.265362+0000 mgr.y (mgr.24475) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:53 smithi155 bash[20805]: cluster 2024-02-18T01:17:52.265362+0000 mgr.y (mgr.24475) 24 : cluster [DBG] pgmap v11: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:54.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.187+0000 7f5c51f82700 1 -- 172.21.15.155:0/1974866127 >> v1:172.21.15.155:6789/0 conn(0x7f5c4c0fc780 legacy=0x7f5c4c0fcb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:54.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.187+0000 7f5c51f82700 1 -- 172.21.15.155:0/1974866127 shutdown_connections 2024-02-18T01:17:54.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.187+0000 7f5c51f82700 1 -- 172.21.15.155:0/1974866127 >> 172.21.15.155:0/1974866127 conn(0x7f5c4c06c760 msgr2=0x7f5c4c06cb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:54.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.187+0000 7f5c51f82700 1 -- 172.21.15.155:0/1974866127 shutdown_connections 2024-02-18T01:17:54.193 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.187+0000 7f5c51f82700 1 -- 172.21.15.155:0/1974866127 wait complete. 2024-02-18T01:17:54.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 Processor -- start 2024-02-18T01:17:54.194 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- start start 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c4c116320 con 0x7f5c4c1030c0 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c4b7fe700 1 --1- >> v1:172.21.15.155:6789/0 conn(0x7f5c4c0fc780 0x7f5c4c0f8ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.155:6789/0 says I am v1:172.21.15.155:37072/0 (socket says 172.21.15.155:37072) 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c4b7fe700 1 -- 172.21.15.155:0/3543907391 learned_addr learned my addr 172.21.15.155:0/3543907391 (peer_addr_for_me v1:172.21.15.155:0/0) 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c4c117520 con 0x7f5c4c0fc780 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5c4c118720 con 0x7f5c4c0ff580 2024-02-18T01:17:54.195 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601647642 0 0) 0x7f5c4c117520 con 0x7f5c4c0fc780 2024-02-18T01:17:54.196 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c34003030 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 659886511 0 0) 0x7f5c34003030 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5c4c117520 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5c3c001430 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1250977226 0 0) 0x7f5c4c118720 con 0x7f5c4c0ff580 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c34003030 con 0x7f5c4c0ff580 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614726328 0 0) 0x7f5c4c116320 con 0x7f5c4c1030c0 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c4c118720 con 0x7f5c4c1030c0 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2284372236 0 0) 0x7f5c4c117520 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 >> v1:172.21.15.105:6790/0 conn(0x7f5c4c0ff580 legacy=0x7f5c4c0f91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 >> v1:172.21.15.105:6789/0 conn(0x7f5c4c1030c0 legacy=0x7f5c4c1149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c4c119920 con 0x7f5c4c0fc780 2024-02-18T01:17:54.197 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c4c118910 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c4c118e00 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f5c3c0016e0 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.191+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5c3c0039b0 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.195+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c38004f50 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.195+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f5c3c003c70 con 0x7f5c4c0fc780 2024-02-18T01:17:54.198 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.195+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7f5c3c078740 con 0x7f5c4c0fc780 2024-02-18T01:17:54.204 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.199+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f5c3c04c370 con 0x7f5c4c0fc780 2024-02-18T01:17:54.456 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.451+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 --> v1:172.21.15.155:6789/0 -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f5c38005780 con 0x7f5c4c0fc780 2024-02-18T01:17:54.465 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.459+0000 7f5c48ff9700 1 -- 172.21.15.155:0/3543907391 <== mon.1 v1:172.21.15.155:6789/0 10 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (unknown 3508017507 0 4213584097) 0x7f5c3c04a7e0 con 0x7f5c4c0fc780 2024-02-18T01:17:54.466 INFO:teuthology.orchestra.run.smithi155.stdout:[client.1] 2024-02-18T01:17:54.466 INFO:teuthology.orchestra.run.smithi155.stdout: key = AQDCWtFlUfBEGxAATOE6uORq8yN69fTPKpS5jw== 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 >> v1:172.21.15.105:6800/2148176012 conn(0x7f5c34064490 legacy=0x7f5c34066950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 >> v1:172.21.15.155:6789/0 conn(0x7f5c4c0fc780 legacy=0x7f5c4c0f8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 shutdown_connections 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 >> 172.21.15.155:0/3543907391 conn(0x7f5c4c06c760 msgr2=0x7f5c4c06d360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 shutdown_connections 2024-02-18T01:17:54.470 INFO:teuthology.orchestra.run.smithi155.stderr:2024-02-18T01:17:54.467+0000 7f5c51f82700 1 -- 172.21.15.155:0/3543907391 wait complete. 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[19734]: audit 2024-02-18T01:17:54.456233+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.155:0/3543907391' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[19734]: audit 2024-02-18T01:17:54.457139+0000 mon.a (mon.0) 653 : audit [INF] from='client.24511 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[19734]: audit 2024-02-18T01:17:54.464460+0000 mon.a (mon.0) 654 : audit [INF] from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[27669]: audit 2024-02-18T01:17:54.456233+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.155:0/3543907391' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[27669]: audit 2024-02-18T01:17:54.457139+0000 mon.a (mon.0) 653 : audit [INF] from='client.24511 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:54 smithi105 bash[27669]: audit 2024-02-18T01:17:54.464460+0000 mon.a (mon.0) 654 : audit [INF] from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:54 smithi155 bash[20805]: audit 2024-02-18T01:17:54.456233+0000 mon.b (mon.1) 26 : audit [INF] from='client.? v1:172.21.15.155:0/3543907391' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:54 smithi155 bash[20805]: audit 2024-02-18T01:17:54.457139+0000 mon.a (mon.0) 653 : audit [INF] from='client.24511 ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-02-18T01:17:54.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:54 smithi155 bash[20805]: audit 2024-02-18T01:17:54.464460+0000 mon.a (mon.0) 654 : audit [INF] from='client.24511 ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-02-18T01:17:55.356 DEBUG:teuthology.orchestra.run.smithi155:> set -ex 2024-02-18T01:17:55.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-02-18T01:17:55.357 DEBUG:teuthology.orchestra.run.smithi155:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-02-18T01:17:55.383 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T01:17:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:55 smithi105 bash[19734]: cluster 2024-02-18T01:17:54.266588+0000 mgr.y (mgr.24475) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:17:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:55 smithi105 bash[27669]: cluster 2024-02-18T01:17:54.266588+0000 mgr.y (mgr.24475) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:17:55.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:55 smithi155 bash[20805]: cluster 2024-02-18T01:17:54.266588+0000 mgr.y (mgr.24475) 25 : cluster [DBG] pgmap v12: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:17:57.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:17:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:17:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:57 smithi105 bash[19734]: cluster 2024-02-18T01:17:56.267315+0000 mgr.y (mgr.24475) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:57 smithi105 bash[27669]: cluster 2024-02-18T01:17:56.267315+0000 mgr.y (mgr.24475) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:57 smithi155 bash[20805]: cluster 2024-02-18T01:17:56.267315+0000 mgr.y (mgr.24475) 26 : cluster [DBG] pgmap v13: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:17:58.867 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:58 smithi155 bash[20805]: audit 2024-02-18T01:17:57.272596+0000 mgr.y (mgr.24475) 27 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:59.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:58 smithi105 bash[19734]: audit 2024-02-18T01:17:57.272596+0000 mgr.y (mgr.24475) 27 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:59.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:58 smithi105 bash[27669]: audit 2024-02-18T01:17:57.272596+0000 mgr.y (mgr.24475) 27 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:17:59.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:17:59 smithi155 bash[20805]: cluster 2024-02-18T01:17:58.268853+0000 mgr.y (mgr.24475) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:17:59.994 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:17:59 smithi155 systemd[1]: Started Ceph prometheus.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:00.211 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:00.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:17:59 smithi105 bash[19734]: cluster 2024-02-18T01:17:58.268853+0000 mgr.y (mgr.24475) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:00.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:17:59 smithi105 bash[27669]: cluster 2024-02-18T01:17:58.268853+0000 mgr.y (mgr.24475) 28 : cluster [DBG] pgmap v14: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.913Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.913Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.913Z caller=main.go:518 level=info host_details="(Linux 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 smithi155 (none))" 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.913Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.913Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.915Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.916Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-18T01:18:00.919 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.917Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-18T01:18:00.920 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[20805]: audit 2024-02-18T01:17:59.782113+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:00.920 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[20805]: cephadm 2024-02-18T01:17:59.799597+0000 mgr.y (mgr.24475) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi105 2024-02-18T01:18:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:00 smithi105 bash[19734]: audit 2024-02-18T01:17:59.782113+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:00 smithi105 bash[19734]: cephadm 2024-02-18T01:17:59.799597+0000 mgr.y (mgr.24475) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi105 2024-02-18T01:18:01.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:00 smithi105 bash[27669]: audit 2024-02-18T01:17:59.782113+0000 mon.a (mon.0) 655 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:01.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:00 smithi105 bash[27669]: cephadm 2024-02-18T01:17:59.799597+0000 mgr.y (mgr.24475) 29 : cephadm [INF] Deploying daemon alertmanager.a on smithi105 2024-02-18T01:18:01.243 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.922Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.922Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.924µs 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.922Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.922Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.922Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=43.823µs wal_replay_duration=272.923µs total_replay_duration=345.081µs 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.924Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.924Z caller=main.go:947 level=info msg="TSDB started" 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.924Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.950Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=26.17313ms db_storage=984ns remote_storage=1.872µs web_handler=474ns query_engine=788ns scrape=374.427µs scrape_sd=38.067µs notify=1.598µs notify_sd=2.432µs rules=25.369817ms 2024-02-18T01:18:01.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:00 smithi155 bash[46533]: ts=2024-02-18T01:18:00.950Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-18T01:18:02.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:01 smithi105 bash[27669]: cluster 2024-02-18T01:18:00.269535+0000 mgr.y (mgr.24475) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:01 smithi105 bash[27669]: audit 2024-02-18T01:18:01.502958+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:01 smithi105 bash[19734]: cluster 2024-02-18T01:18:00.269535+0000 mgr.y (mgr.24475) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:01 smithi105 bash[19734]: audit 2024-02-18T01:18:01.502958+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:01 smithi155 bash[20805]: cluster 2024-02-18T01:18:00.269535+0000 mgr.y (mgr.24475) 30 : cluster [DBG] pgmap v15: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:01 smithi155 bash[20805]: audit 2024-02-18T01:18:01.502958+0000 mon.a (mon.0) 656 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- 172.21.15.105:0/943473718 >> v1:172.21.15.105:6790/0 conn(0x7f0b000fc970 legacy=0x7f0b000fcd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- 172.21.15.105:0/943473718 shutdown_connections 2024-02-18T01:18:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- 172.21.15.105:0/943473718 >> 172.21.15.105:0/943473718 conn(0x7f0b000f2570 msgr2=0x7f0b000f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- 172.21.15.105:0/943473718 shutdown_connections 2024-02-18T01:18:04.099 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- 172.21.15.105:0/943473718 wait complete. 2024-02-18T01:18:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 Processor -- start 2024-02-18T01:18:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- start start 2024-02-18T01:18:04.100 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b001161b0 con 0x7f0b001032b0 2024-02-18T01:18:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b00117390 con 0x7f0b000ff770 2024-02-18T01:18:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0b061e5700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0b00118570 con 0x7f0b000fc970 2024-02-18T01:18:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0aff7fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f0b000fc970 0x7f0b000f8760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:50370/0 (socket says 172.21.15.105:50370) 2024-02-18T01:18:04.101 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0aff7fe700 1 -- 172.21.15.105:0/869988490 learned_addr learned my addr 172.21.15.105:0/869988490 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2782387726 0 0) 0x7f0b00118570 con 0x7f0b000fc970 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aec003110 con 0x7f0b000fc970 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2496530451 0 0) 0x7f0b001161b0 con 0x7f0b001032b0 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00118570 con 0x7f0b001032b0 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2783518675 0 0) 0x7f0aec003110 con 0x7f0b000fc970 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0b001161b0 con 0x7f0b000fc970 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0ae8002be0 con 0x7f0b000fc970 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2006435152 0 0) 0x7f0b00118570 con 0x7f0b001032b0 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f0aec003110 con 0x7f0b001032b0 2024-02-18T01:18:04.103 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.095+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081447573 0 0) 0x7f0b00117390 con 0x7f0b000ff770 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b00118570 con 0x7f0b000ff770 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0af8001d70 con 0x7f0b001032b0 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2559044339 0 0) 0x7f0b001161b0 con 0x7f0b000fc970 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 >> v1:172.21.15.155:6789/0 conn(0x7f0b000ff770 legacy=0x7f0b000f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 >> v1:172.21.15.105:6789/0 conn(0x7f0b001032b0 legacy=0x7f0b00114930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00119770 con 0x7f0b000fc970 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b001163a0 con 0x7f0b000fc970 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b00116810 con 0x7f0b000fc970 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f0ae8002ea0 con 0x7f0b000fc970 2024-02-18T01:18:04.104 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0ae80038a0 con 0x7f0b000fc970 2024-02-18T01:18:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ae0004f20 con 0x7f0b000fc970 2024-02-18T01:18:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f0ae8003140 con 0x7f0b000fc970 2024-02-18T01:18:04.105 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.099+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7f0ae8078930 con 0x7f0b000fc970 2024-02-18T01:18:04.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.107+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f0ae804c5c0 con 0x7f0b000fc970 2024-02-18T01:18:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:03 smithi105 bash[27669]: cluster 2024-02-18T01:18:02.270503+0000 mgr.y (mgr.24475) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:03 smithi105 bash[19734]: cluster 2024-02-18T01:18:02.270503+0000 mgr.y (mgr.24475) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:03 smithi155 bash[20805]: cluster 2024-02-18T01:18:02.270503+0000 mgr.y (mgr.24475) 31 : cluster [DBG] pgmap v16: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:04.290 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.287+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ae00051d0 con 0x7f0b000fc970 2024-02-18T01:18:04.293 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.287+0000 7f0afcff9700 1 -- 172.21.15.105:0/869988490 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+19063 (unknown 1763841132 0 3308992677) 0x7f0ae804aa30 con 0x7f0b000fc970 2024-02-18T01:18:04.293 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:18:04.294 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":62,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","created":"2024-02-18T01:08:57.371365+0000","modified":"2024-02-18T01:17:36.238124+0000","last_up_change":"2024-02-18T01:16:55.358934+0000","last_in_change":"2024-02-18T01:16:29.814709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T01:13:29.045566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-18T01:17:08.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-18T01:17:09.343824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T01:17:11.795816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T01:17:13.251513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T01:17:15.371360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"62250a45-b3fb-4078-8edb-dc54138a7b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6801","nonce":3115838352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6802","nonce":3115838352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6804","nonce":3115838352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6803","nonce":3115838352}]},"public_addr":"172.21.15.105:6801/3115838352","cluster_addr":"172.21.15.105:6802/3115838352","heartbeat_back_addr":"172.21.15.105:6804/3115838352","heartbeat_front_addr":"172.21.15.105:6803/3115838352","state":["exists","up"]},{"osd":1,"uuid":"59487fa0-277f-4175-b49b-f50ee2645931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6805","nonce":1532722979}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6806","nonce":1532722979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6808","nonce":1532722979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6807","nonce":1532722979}]},"public_addr":"172.21.15.105:6805/1532722979","cluster_addr":"172.21.15.105:6806/1532722979","heartbeat_back_addr":"172.21.15.105:6808/1532722979","heartbeat_front_addr":"172.21.15.105:6807/1532722979","state":["exists","up"]},{"osd":2,"uuid":"c08740fc-b607-4226-be20-1c56854e971c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6809","nonce":3935238244}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6810","nonce":3935238244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6812","nonce":3935238244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6811","nonce":3935238244}]},"public_addr":"172.21.15.105:6809/3935238244","cluster_addr":"172.21.15.105:6810/3935238244","heartbeat_back_addr":"172.21.15.105:6812/3935238244","heartbeat_front_addr":"172.21.15.105:6811/3935238244","state":["exists","up"]},{"osd":3,"uuid":"eaef36ae-a5ea-4a4a-8c5f-c88098a48611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6813","nonce":1099653553}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6814","nonce":1099653553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6816","nonce":1099653553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6815","nonce":1099653553}]},"public_addr":"172.21.15.105:6813/1099653553","cluster_addr":"172.21.15.105:6814/1099653553","heartbeat_back_addr":"172.21.15.105:6816/1099653553","heartbeat_front_addr":"172.21.15.105:6815/1099653553","state":["exists","up"]},{"osd":4,"uuid":"21226459-02ce-41a2-8761-c8125feea431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1608438726}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1608438726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1608438726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1608438726}]},"public_addr":"172.21.15.155:6800/1608438726","cluster_addr":"172.21.15.155:6801/1608438726","heartbeat_back_addr":"172.21.15.155:6803/1608438726","heartbeat_front_addr":"172.21.15.155:6802/1608438726","state":["exists","up"]},{"osd":5,"uuid":"2650bfc0-0a69-4425-aaef-31bada765f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":3674587623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":3674587623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":3674587623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":3674587623}]},"public_addr":"172.21.15.155:6804/3674587623","cluster_addr":"172.21.15.155:6805/3674587623","heartbeat_back_addr":"172.21.15.155:6807/3674587623","heartbeat_front_addr":"172.21.15.155:6806/3674587623","state":["exists","up"]},{"osd":6,"uuid":"565ca7a5-3796-4f8a-93be-8e98f818ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":98819544}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":98819544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":98819544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":98819544}]},"public_addr":"172.21.15.155:6808/98819544","cluster_addr":"172.21.15.155:6809/98819544","heartbeat_back_addr":"172.21.15.155:6811/98819544","heartbeat_front_addr":"172.21.15.155:6810/98819544","state":["exists","up"]},{"osd":7,"uuid":"0b9beb5b-519d-4ef0-9768-cf509db5d07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":1643073216}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":1643073216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":1643073216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":1643073216}]},"public_addr":"172.21.15.155:6812/1643073216","cluster_addr":"172.21.15.155:6813/1643073216","heartbeat_back_addr":"172.21.15.155:6815/1643073216","heartbeat_front_addr":"172.21.15.155:6814/1643073216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:03.400874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:44.855453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:13:26.607272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:09.302207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:48.550664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:15:30.305002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:11.430016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:53.541617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6800/3087430119":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/4165234016":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/3637238460":"2024-02-19T01:09:55.434898+0000","172.21.15.105:0/2779353230":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/1813807489":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/3260915113":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/2780429744":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/1333885900":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/2401194680":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/999225556":"2024-02-19T01:09:25.603699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 >> v1:172.21.15.105:6800/2148176012 conn(0x7f0aec062530 legacy=0x7f0aec0649f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 >> v1:172.21.15.105:6790/0 conn(0x7f0b000fc970 legacy=0x7f0b000f8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 shutdown_connections 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 >> 172.21.15.105:0/869988490 conn(0x7f0b000f2570 msgr2=0x7f0b00106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 shutdown_connections 2024-02-18T01:18:04.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:04.291+0000 7f0b061e5700 1 -- 172.21.15.105:0/869988490 wait complete. 2024-02-18T01:18:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:04 smithi105 bash[19734]: audit 2024-02-18T01:18:04.290530+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? v1:172.21.15.105:0/869988490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:18:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:04 smithi105 bash[27669]: audit 2024-02-18T01:18:04.290530+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? v1:172.21.15.105:0/869988490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:18:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:04 smithi155 bash[20805]: audit 2024-02-18T01:18:04.290530+0000 mon.c (mon.2) 23 : audit [DBG] from='client.? v1:172.21.15.105:0/869988490' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:18:05.518 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2024-02-18T01:13:29.045566+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'iscsi', 'create_time': '2024-02-18T01:17:08.301344+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 3, 'pg_placement_num': 3, 'pg_placement_num_target': 3, 'pg_num_target': 3, 'pg_num_pending': 3, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': '.rgw.root', 'create_time': '2024-02-18T01:17:09.343824+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '55', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 4, 'pool_name': 'default.rgw.log', 'create_time': '2024-02-18T01:17:11.795816+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '57', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 5, 'pool_name': 'default.rgw.control', 'create_time': '2024-02-18T01:17:13.251513+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '59', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rgw': {}}}, {'pool': 6, 'pool_name': 'default.rgw.meta', 'create_time': '2024-02-18T01:17:15.371360+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4}, 'application_metadata': {'rgw': {}}}] 2024-02-18T01:18:05.519 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get device_health_metrics pg_num 2024-02-18T01:18:06.130 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:05 smithi105 bash[27669]: cluster 2024-02-18T01:18:04.270972+0000 mgr.y (mgr.24475) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:06.130 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:05 smithi105 bash[19734]: cluster 2024-02-18T01:18:04.270972+0000 mgr.y (mgr.24475) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:05 smithi155 bash[20805]: cluster 2024-02-18T01:18:04.270972+0000 mgr.y (mgr.24475) 32 : cluster [DBG] pgmap v17: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:06.994 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:06] "GET /metrics HTTP/1.1" 200 172131 "" "Prometheus/2.33.4" 2024-02-18T01:18:07.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:08.188 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:07 smithi105 bash[27669]: cluster 2024-02-18T01:18:06.271636+0000 mgr.y (mgr.24475) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:08.188 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:07 smithi105 bash[19734]: cluster 2024-02-18T01:18:06.271636+0000 mgr.y (mgr.24475) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:08.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:07 smithi155 bash[20805]: cluster 2024-02-18T01:18:06.271636+0000 mgr.y (mgr.24475) 33 : cluster [DBG] pgmap v18: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:09.062 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:08 smithi105 bash[19734]: audit 2024-02-18T01:18:07.283293+0000 mgr.y (mgr.24475) 34 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:09.062 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:08 smithi105 bash[27669]: audit 2024-02-18T01:18:07.283293+0000 mgr.y (mgr.24475) 34 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:08 smithi155 bash[20805]: audit 2024-02-18T01:18:07.283293+0000 mgr.y (mgr.24475) 34 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:09.387 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:09 smithi105 systemd[1]: Started Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:09.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE Bus STOPPING 2024-02-18T01:18:09.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T01:18:09.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE Bus STOPPED 2024-02-18T01:18:09.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE Bus STARTING 2024-02-18T01:18:09.741 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE Serving on http://:::9283 2024-02-18T01:18:09.741 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[20046]: [18/Feb/2024:01:18:09] ENGINE Bus STARTED 2024-02-18T01:18:10.212 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: cluster 2024-02-18T01:18:08.273118+0000 mgr.y (mgr.24475) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:10.212 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: audit 2024-02-18T01:18:09.099577+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: audit 2024-02-18T01:18:09.350388+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: audit 2024-02-18T01:18:09.358017+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: audit 2024-02-18T01:18:09.365239+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[19734]: audit 2024-02-18T01:18:09.371021+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: cluster 2024-02-18T01:18:08.273118+0000 mgr.y (mgr.24475) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: audit 2024-02-18T01:18:09.099577+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: audit 2024-02-18T01:18:09.350388+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: audit 2024-02-18T01:18:09.358017+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: audit 2024-02-18T01:18:09.365239+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:10.213 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:09 smithi105 bash[27669]: audit 2024-02-18T01:18:09.371021+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: cluster 2024-02-18T01:18:08.273118+0000 mgr.y (mgr.24475) 35 : cluster [DBG] pgmap v19: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: audit 2024-02-18T01:18:09.099577+0000 mon.a (mon.0) 657 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: audit 2024-02-18T01:18:09.350388+0000 mon.a (mon.0) 658 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: audit 2024-02-18T01:18:09.358017+0000 mon.a (mon.0) 659 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: audit 2024-02-18T01:18:09.365239+0000 mon.a (mon.0) 660 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:10.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:09 smithi155 bash[20805]: audit 2024-02-18T01:18:09.371021+0000 mon.a (mon.0) 661 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:10.348 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:10.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.446Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-18T01:18:10.490 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.446Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-18T01:18:10.490 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.450Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.105 port=9094 2024-02-18T01:18:10.490 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.452Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-18T01:18:10.837 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.502Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T01:18:10.837 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.502Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T01:18:10.837 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.505Z caller=main.go:518 msg=Listening address=:9093 2024-02-18T01:18:10.837 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:10.506Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-18T01:18:10.838 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[19734]: audit 2024-02-18T01:18:09.365896+0000 mgr.y (mgr.24475) 36 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:10.838 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[19734]: cephadm 2024-02-18T01:18:09.378156+0000 mgr.y (mgr.24475) 37 : cephadm [INF] Deploying daemon grafana.a on smithi155 2024-02-18T01:18:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[27669]: audit 2024-02-18T01:18:09.365896+0000 mgr.y (mgr.24475) 36 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:10 smithi105 bash[27669]: cephadm 2024-02-18T01:18:09.378156+0000 mgr.y (mgr.24475) 37 : cephadm [INF] Deploying daemon grafana.a on smithi155 2024-02-18T01:18:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:10 smithi155 bash[20805]: audit 2024-02-18T01:18:09.365896+0000 mgr.y (mgr.24475) 36 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-02-18T01:18:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:10 smithi155 bash[20805]: cephadm 2024-02-18T01:18:09.378156+0000 mgr.y (mgr.24475) 37 : cephadm [INF] Deploying daemon grafana.a on smithi155 2024-02-18T01:18:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:11 smithi105 bash[19734]: cluster 2024-02-18T01:18:10.274106+0000 mgr.y (mgr.24475) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:11 smithi105 bash[19734]: audit 2024-02-18T01:18:11.515410+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:11 smithi105 bash[27669]: cluster 2024-02-18T01:18:10.274106+0000 mgr.y (mgr.24475) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:11 smithi105 bash[27669]: audit 2024-02-18T01:18:11.515410+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:11 smithi155 bash[20805]: cluster 2024-02-18T01:18:10.274106+0000 mgr.y (mgr.24475) 38 : cluster [DBG] pgmap v20: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:11 smithi155 bash[20805]: audit 2024-02-18T01:18:11.515410+0000 mon.a (mon.0) 662 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:18:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:18:12.739 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:12 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:12.453Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000798396s 2024-02-18T01:18:13.622 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/147528167 >> v1:172.21.15.105:6790/0 conn(0x7ffb14106700 legacy=0x7ffb14108af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/147528167 shutdown_connections 2024-02-18T01:18:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/147528167 >> 172.21.15.105:0/147528167 conn(0x7ffb140f2530 msgr2=0x7ffb140f4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/147528167 shutdown_connections 2024-02-18T01:18:13.623 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/147528167 wait complete. 2024-02-18T01:18:13.624 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 Processor -- start 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- start start 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb141196e0 con 0x7ffb140f7320 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb1411a8c0 con 0x7ffb140f6830 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb1a5a5700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ffb1411bac0 con 0x7ffb14106700 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb137fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7ffb140f7320 0x7ffb141022f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:48646/0 (socket says 172.21.15.105:48646) 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb137fe700 1 -- 172.21.15.105:0/1294555646 learned_addr learned my addr 172.21.15.105:0/1294555646 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 550843251 0 0) 0x7ffb1411bac0 con 0x7ffb14106700 2024-02-18T01:18:13.625 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffafc0033f0 con 0x7ffb14106700 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266171283 0 0) 0x7ffb141196e0 con 0x7ffb140f7320 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb1411bac0 con 0x7ffb140f7320 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2570607154 0 0) 0x7ffb1411a8c0 con 0x7ffb140f6830 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb141196e0 con 0x7ffb140f6830 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 710081666 0 0) 0x7ffafc0033f0 con 0x7ffb14106700 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb1411a8c0 con 0x7ffb14106700 2024-02-18T01:18:13.626 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ffb0c002c60 con 0x7ffb14106700 2024-02-18T01:18:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 635305171 0 0) 0x7ffb1411bac0 con 0x7ffb140f7320 2024-02-18T01:18:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffafc0033f0 con 0x7ffb140f7320 2024-02-18T01:18:13.628 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1854784101 0 0) 0x7ffb141196e0 con 0x7ffb140f6830 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ffb1411bac0 con 0x7ffb140f6830 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ffb08001980 con 0x7ffb140f7320 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ffb04001bc0 con 0x7ffb140f6830 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1408942829 0 0) 0x7ffb1411a8c0 con 0x7ffb14106700 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 >> v1:172.21.15.155:6789/0 conn(0x7ffb140f6830 legacy=0x7ffb14101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 >> v1:172.21.15.105:6789/0 conn(0x7ffb140f7320 legacy=0x7ffb141022f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.619+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1411ccc0 con 0x7ffb14106700 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1411bcb0 con 0x7ffb14106700 2024-02-18T01:18:13.629 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb1411c120 con 0x7ffb14106700 2024-02-18T01:18:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7ffb0c001bb0 con 0x7ffb14106700 2024-02-18T01:18:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ffb0c003680 con 0x7ffb14106700 2024-02-18T01:18:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb00004f20 con 0x7ffb14106700 2024-02-18T01:18:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7ffb0c0165d0 con 0x7ffb14106700 2024-02-18T01:18:13.630 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.623+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7ffb0c076060 con 0x7ffb14106700 2024-02-18T01:18:13.639 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.635+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ffb0c049c90 con 0x7ffb14106700 2024-02-18T01:18:13.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.819+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7ffb00005750 con 0x7ffb14106700 2024-02-18T01:18:13.824 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.819+0000 7ffb117fa700 1 -- 172.21.15.105:0/1294555646 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v62) v1 ==== 110+0+10 (unknown 3201642675 0 2170607528) 0x7ffb0c048100 con 0x7ffb14106700 2024-02-18T01:18:13.824 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 1 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 >> v1:172.21.15.105:6800/2148176012 conn(0x7ffafc0623c0 legacy=0x7ffafc064880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 >> v1:172.21.15.105:6790/0 conn(0x7ffb14106700 legacy=0x7ffb14117da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 shutdown_connections 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 >> 172.21.15.105:0/1294555646 conn(0x7ffb140f2530 msgr2=0x7ffb140f5920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 shutdown_connections 2024-02-18T01:18:13.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:13.823+0000 7ffb1a5a5700 1 -- 172.21.15.105:0/1294555646 wait complete. 2024-02-18T01:18:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:13 smithi105 bash[19734]: cluster 2024-02-18T01:18:12.275609+0000 mgr.y (mgr.24475) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:13 smithi105 bash[19734]: audit 2024-02-18T01:18:13.823315+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? v1:172.21.15.105:0/1294555646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T01:18:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:13 smithi105 bash[27669]: cluster 2024-02-18T01:18:12.275609+0000 mgr.y (mgr.24475) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:14.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:13 smithi105 bash[27669]: audit 2024-02-18T01:18:13.823315+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? v1:172.21.15.105:0/1294555646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T01:18:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:13 smithi155 bash[20805]: cluster 2024-02-18T01:18:12.275609+0000 mgr.y (mgr.24475) 39 : cluster [DBG] pgmap v21: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:13 smithi155 bash[20805]: audit 2024-02-18T01:18:13.823315+0000 mon.c (mon.2) 24 : audit [DBG] from='client.? v1:172.21.15.105:0/1294555646' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2024-02-18T01:18:14.468 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get iscsi pg_num 2024-02-18T01:18:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:15 smithi105 bash[19734]: cluster 2024-02-18T01:18:14.276352+0000 mgr.y (mgr.24475) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:15 smithi105 bash[27669]: cluster 2024-02-18T01:18:14.276352+0000 mgr.y (mgr.24475) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:15 smithi155 bash[20805]: cluster 2024-02-18T01:18:14.276352+0000 mgr.y (mgr.24475) 40 : cluster [DBG] pgmap v22: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:16] "GET /metrics HTTP/1.1" 200 172131 "" "Prometheus/2.33.4" 2024-02-18T01:18:17.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:17 smithi105 bash[19734]: cluster 2024-02-18T01:18:16.277086+0000 mgr.y (mgr.24475) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:17 smithi105 bash[27669]: cluster 2024-02-18T01:18:16.277086+0000 mgr.y (mgr.24475) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:17 smithi155 bash[20805]: cluster 2024-02-18T01:18:16.277086+0000 mgr.y (mgr.24475) 41 : cluster [DBG] pgmap v23: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:18 smithi105 bash[27669]: audit 2024-02-18T01:18:17.285001+0000 mgr.y (mgr.24475) 42 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:18 smithi105 bash[19734]: audit 2024-02-18T01:18:17.285001+0000 mgr.y (mgr.24475) 42 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:18 smithi155 bash[20805]: audit 2024-02-18T01:18:17.285001+0000 mgr.y (mgr.24475) 42 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:19.306 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:20.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:19 smithi105 bash[27669]: cluster 2024-02-18T01:18:18.278551+0000 mgr.y (mgr.24475) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:20.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:19 smithi105 bash[19734]: cluster 2024-02-18T01:18:18.278551+0000 mgr.y (mgr.24475) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:19 smithi155 bash[20805]: cluster 2024-02-18T01:18:18.278551+0000 mgr.y (mgr.24475) 43 : cluster [DBG] pgmap v24: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:20.580 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:20 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:20.456Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003852728s 2024-02-18T01:18:22.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.158+0000 7f95d3c8f700 1 -- 172.21.15.105:0/833354644 >> v1:172.21.15.105:6789/0 conn(0x7f95cc0f73c0 legacy=0x7f95cc109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:22.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.158+0000 7f95d3c8f700 1 -- 172.21.15.105:0/833354644 shutdown_connections 2024-02-18T01:18:22.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.158+0000 7f95d3c8f700 1 -- 172.21.15.105:0/833354644 >> 172.21.15.105:0/833354644 conn(0x7f95cc0f2570 msgr2=0x7f95cc0f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:22.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.158+0000 7f95d3c8f700 1 -- 172.21.15.105:0/833354644 shutdown_connections 2024-02-18T01:18:22.165 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.158+0000 7f95d3c8f700 1 -- 172.21.15.105:0/833354644 wait complete. 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 Processor -- start 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- start start 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95cc11d9c0 con 0x7f95cc0f6810 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95cc11eba0 con 0x7f95cc0f73c0 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f95cc11fda0 con 0x7f95cc0fca50 2024-02-18T01:18:22.166 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d222c700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f95cc0fca50 0x7f95cc11c080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:33536/0 (socket says 172.21.15.105:33536) 2024-02-18T01:18:22.167 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d222c700 1 -- 172.21.15.105:0/3803486830 learned_addr learned my addr 172.21.15.105:0/3803486830 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459254805 0 0) 0x7f95cc11d9c0 con 0x7f95cc0f6810 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b80033f0 con 0x7f95cc0f6810 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071267505 0 0) 0x7f95cc11fda0 con 0x7f95cc0fca50 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc11d9c0 con 0x7f95cc0fca50 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1029926264 0 0) 0x7f95cc11eba0 con 0x7f95cc0f73c0 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95cc11fda0 con 0x7f95cc0f73c0 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3442038858 0 0) 0x7f95b80033f0 con 0x7f95cc0f6810 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95cc11eba0 con 0x7f95cc0f6810 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3680738649 0 0) 0x7f95cc11d9c0 con 0x7f95cc0fca50 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f95b80033f0 con 0x7f95cc0fca50 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f95b4001be0 con 0x7f95cc0f6810 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f95c4001800 con 0x7f95cc0fca50 2024-02-18T01:18:22.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 979149361 0 0) 0x7f95cc11eba0 con 0x7f95cc0f6810 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 >> v1:172.21.15.105:6790/0 conn(0x7f95cc0fca50 legacy=0x7f95cc11c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 >> v1:172.21.15.155:6789/0 conn(0x7f95cc0f73c0 legacy=0x7f95cc107800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95cc120fa0 con 0x7f95cc0f6810 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc11ff90 con 0x7f95cc0f6810 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc120400 con 0x7f95cc0f6810 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f95b40022f0 con 0x7f95cc0f6810 2024-02-18T01:18:22.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f95b4003620 con 0x7f95cc0f6810 2024-02-18T01:18:22.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.162+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac004f20 con 0x7f95cc0f6810 2024-02-18T01:18:22.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.166+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f95b4016590 con 0x7f95cc0f6810 2024-02-18T01:18:22.171 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.166+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7f95b4076030 con 0x7f95cc0f6810 2024-02-18T01:18:22.178 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.174+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f95b4049db0 con 0x7f95cc0f6810 2024-02-18T01:18:22.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:21 smithi105 bash[27669]: cluster 2024-02-18T01:18:20.279312+0000 mgr.y (mgr.24475) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:21 smithi105 bash[19734]: cluster 2024-02-18T01:18:20.279312+0000 mgr.y (mgr.24475) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:21 smithi155 bash[20805]: cluster 2024-02-18T01:18:20.279312+0000 mgr.y (mgr.24475) 44 : cluster [DBG] pgmap v25: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:18:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:18:22.361 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.354+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"} v 0) v1 -- 0x7f95ac005750 con 0x7f95cc0f6810 2024-02-18T01:18:22.362 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95caffd700 1 -- 172.21.15.105:0/3803486830 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]=0 v62) v1 ==== 94+0+10 (unknown 3507771597 0 2787501894) 0x7f95b40481c0 con 0x7f95cc0f6810 2024-02-18T01:18:22.363 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 3 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 >> v1:172.21.15.105:6800/2148176012 conn(0x7f95b8062200 legacy=0x7f95b80646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 >> v1:172.21.15.105:6789/0 conn(0x7f95cc0f6810 legacy=0x7f95cc1070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 shutdown_connections 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 >> 172.21.15.105:0/3803486830 conn(0x7f95cc0f2570 msgr2=0x7f95cc0f4960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 shutdown_connections 2024-02-18T01:18:22.365 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:22.358+0000 7f95d3c8f700 1 -- 172.21.15.105:0/3803486830 wait complete. 2024-02-18T01:18:23.021 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get .rgw.root pg_num 2024-02-18T01:18:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:22 smithi105 bash[27669]: audit 2024-02-18T01:18:22.361543+0000 mon.a (mon.0) 663 : audit [DBG] from='client.? v1:172.21.15.105:0/3803486830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T01:18:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:22 smithi105 bash[19734]: audit 2024-02-18T01:18:22.361543+0000 mon.a (mon.0) 663 : audit [DBG] from='client.? v1:172.21.15.105:0/3803486830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T01:18:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:22 smithi155 bash[20805]: audit 2024-02-18T01:18:22.361543+0000 mon.a (mon.0) 663 : audit [DBG] from='client.? v1:172.21.15.105:0/3803486830' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "iscsi", "var": "pg_num"}]: dispatch 2024-02-18T01:18:24.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:23 smithi105 bash[19734]: cluster 2024-02-18T01:18:22.280683+0000 mgr.y (mgr.24475) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:24.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:23 smithi105 bash[27669]: cluster 2024-02-18T01:18:22.280683+0000 mgr.y (mgr.24475) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:23 smithi155 bash[20805]: cluster 2024-02-18T01:18:22.280683+0000 mgr.y (mgr.24475) 45 : cluster [DBG] pgmap v26: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:25 smithi155 bash[20805]: cluster 2024-02-18T01:18:24.281549+0000 mgr.y (mgr.24475) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:25 smithi105 bash[19734]: cluster 2024-02-18T01:18:24.281549+0000 mgr.y (mgr.24475) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:25 smithi105 bash[27669]: cluster 2024-02-18T01:18:24.281549+0000 mgr.y (mgr.24475) 46 : cluster [DBG] pgmap v27: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:26] "GET /metrics HTTP/1.1" 200 172123 "" "Prometheus/2.33.4" 2024-02-18T01:18:27.370 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:27.858 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:28.235 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:27 smithi155 bash[20805]: cluster 2024-02-18T01:18:26.282256+0000 mgr.y (mgr.24475) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:28.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:27 smithi105 bash[19734]: cluster 2024-02-18T01:18:26.282256+0000 mgr.y (mgr.24475) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:27 smithi105 bash[27669]: cluster 2024-02-18T01:18:26.282256+0000 mgr.y (mgr.24475) 47 : cluster [DBG] pgmap v28: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:28.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:28 smithi155 systemd[1]: Started Ceph grafana.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:29.118 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[27669]: audit 2024-02-18T01:18:27.294108+0000 mgr.y (mgr.24475) 48 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:29.118 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[27669]: audit 2024-02-18T01:18:28.278628+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:29.119 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[27669]: audit 2024-02-18T01:18:28.280592+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:29.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[19734]: audit 2024-02-18T01:18:27.294108+0000 mgr.y (mgr.24475) 48 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:29.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[19734]: audit 2024-02-18T01:18:28.278628+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:29.119 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:28 smithi105 bash[19734]: audit 2024-02-18T01:18:28.280592+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:28 smithi155 bash[20805]: audit 2024-02-18T01:18:27.294108+0000 mgr.y (mgr.24475) 48 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:28 smithi155 bash[20805]: audit 2024-02-18T01:18:28.278628+0000 mon.a (mon.0) 664 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:28 smithi155 bash[20805]: audit 2024-02-18T01:18:28.280592+0000 mon.a (mon.0) 665 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:29.573 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="The state of unified alerting is still not defined. The decision will be made during as we run the database migrations" logger=settings 2024-02-18T01:18:29.573 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=warn msg="falling back to legacy setting of 'min_interval_seconds'; please use the configuration option in the `unified_alerting` section if Grafana 8 alerts are enabled." logger=settings 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config loaded from" logger=settings file=/usr/share/grafana/conf/defaults.ini 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config loaded from" logger=settings file=/etc/grafana/grafana.ini 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_DATA=/var/lib/grafana" 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_LOGS=/var/log/grafana" 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Config overridden from Environment variable" logger=settings var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Path Home" logger=settings path=/usr/share/grafana 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Path Data" logger=settings path=/var/lib/grafana 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Path Logs" logger=settings path=/var/log/grafana 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Path Plugins" logger=settings path=/var/lib/grafana/plugins 2024-02-18T01:18:29.574 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Path Provisioning" logger=settings path=/etc/grafana/provisioning 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="App mode production" logger=settings 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Connecting to DB" logger=sqlstore dbtype=sqlite3 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=warn msg="SQLite database file has broader permissions than it should" logger=sqlstore path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Starting DB migrations" logger=migrator 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="create migration_log table" 2024-02-18T01:18:29.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="create user table" 2024-02-18T01:18:29.858 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.login" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user.email" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_login - v1" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_user_email - v1" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table user to user_v1 - v1" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="create user table v2" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_login - v2" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_user_email - v2" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-18T01:18:29.859 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table user_v1" 2024-02-18T01:18:30.119 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[20805]: cluster 2024-02-18T01:18:28.283276+0000 mgr.y (mgr.24475) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:30.119 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add column help_flags1 to user table" 2024-02-18T01:18:30.119 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Update user table charset" 2024-02-18T01:18:30.119 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add last_seen_at column to user" 2024-02-18T01:18:30.119 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:29 smithi155 bash[46955]: t=2024-02-18T01:18:29+0000 lvl=info msg="Executing migration" logger=migrator id="Add missing user data" 2024-02-18T01:18:30.119 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_disabled column to user" 2024-02-18T01:18:30.120 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add index user.login/user.email" 2024-02-18T01:18:30.120 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Add is_service_account column to user" 2024-02-18T01:18:30.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:29 smithi105 bash[19734]: cluster 2024-02-18T01:18:28.283276+0000 mgr.y (mgr.24475) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:29 smithi105 bash[27669]: cluster 2024-02-18T01:18:28.283276+0000 mgr.y (mgr.24475) 49 : cluster [DBG] pgmap v29: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:30.380 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create temp user table v1-7" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v1-7" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v1-7" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v1-7" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v1-7" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Update temp_user table charset" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_email - v1" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_org_id - v1" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_code - v1" 2024-02-18T01:18:30.381 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_temp_user_status - v1" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create temp_user v2" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_email - v2" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_org_id - v2" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_code - v2" 2024-02-18T01:18:30.650 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_temp_user_status - v2" 2024-02-18T01:18:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/2512996271 >> v1:172.21.15.105:6789/0 conn(0x7fb0840f8a40 legacy=0x7fb0840f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/2512996271 shutdown_connections 2024-02-18T01:18:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/2512996271 >> 172.21.15.105:0/2512996271 conn(0x7fb0840f2570 msgr2=0x7fb0840f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/2512996271 shutdown_connections 2024-02-18T01:18:30.862 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/2512996271 wait complete. 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 Processor -- start 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- start start 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb084119560 con 0x7fb0840f95f0 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb08411a700 con 0x7fb0841043d0 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fb08411b8a0 con 0x7fb0840f8a40 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0837fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fb0840f95f0 0x7fb0840f7100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:39872/0 (socket says 172.21.15.105:39872) 2024-02-18T01:18:30.863 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0837fe700 1 -- 172.21.15.105:0/3248415601 learned_addr learned my addr 172.21.15.105:0/3248415601 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:30.864 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1155891981 0 0) 0x7fb084119560 con 0x7fb0840f95f0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0700033f0 con 0x7fb0840f95f0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050401499 0 0) 0x7fb08411b8a0 con 0x7fb0840f8a40 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb084119560 con 0x7fb0840f8a40 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929741493 0 0) 0x7fb08411a700 con 0x7fb0841043d0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb08411b8a0 con 0x7fb0841043d0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 962228255 0 0) 0x7fb084119560 con 0x7fb0840f8a40 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb08411a700 con 0x7fb0840f8a40 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb074001250 con 0x7fb0840f8a40 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 821781184 0 0) 0x7fb0700033f0 con 0x7fb0840f95f0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb084119560 con 0x7fb0840f95f0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb06c0018d0 con 0x7fb0840f95f0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1482075553 0 0) 0x7fb08411b8a0 con 0x7fb0841043d0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fb0700033f0 con 0x7fb0841043d0 2024-02-18T01:18:30.865 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb07c0017c0 con 0x7fb0841043d0 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3358970995 0 0) 0x7fb08411a700 con 0x7fb0840f8a40 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 >> v1:172.21.15.155:6789/0 conn(0x7fb0841043d0 legacy=0x7fb084117e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 >> v1:172.21.15.105:6789/0 conn(0x7fb0840f95f0 legacy=0x7fb0840f7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08411ca40 con 0x7fb0840f8a40 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08411ba90 con 0x7fb0840f8a40 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb08a9c3700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08411bf00 con 0x7fb0840f8a40 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fb074001e70 con 0x7fb0840f8a40 2024-02-18T01:18:30.866 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.858+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fb074003b30 con 0x7fb0840f8a40 2024-02-18T01:18:30.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.862+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fb074004de0 con 0x7fb0840f8a40 2024-02-18T01:18:30.867 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.862+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb068004f20 con 0x7fb0840f8a40 2024-02-18T01:18:30.868 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.862+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(62..62 src has 1..62) v4 ==== 6056+0+0 (unknown 3356227159 0 0) 0x7fb074076550 con 0x7fb0840f8a40 2024-02-18T01:18:30.875 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:30.870+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fb07404a120 con 0x7fb0840f8a40 2024-02-18T01:18:30.902 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="copy temp_user v1 to v2" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="drop temp_user_tmp_qwerty" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="Set created for temp users that will otherwise prematurely expire" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create star table" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index star.user_id_dashboard_id" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create org table v1" 2024-02-18T01:18:30.903 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_name - v1" 2024-02-18T01:18:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.054+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"} v 0) v1 -- 0x7fb068005750 con 0x7fb0840f8a40 2024-02-18T01:18:31.059 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.054+0000 7fb0817fa700 1 -- 172.21.15.105:0/3248415601 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]=0 v62) v1 ==== 98+0+11 (unknown 255814548 0 3519293104) 0x7fb074048590 con 0x7fb0840f8a40 2024-02-18T01:18:31.059 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-02-18T01:18:31.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.058+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 >> v1:172.21.15.105:6800/2148176012 conn(0x7fb070062310 legacy=0x7fb0700647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:31.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.058+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 >> v1:172.21.15.105:6790/0 conn(0x7fb0840f8a40 legacy=0x7fb0840f69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:31.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.058+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 shutdown_connections 2024-02-18T01:18:31.065 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.058+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 >> 172.21.15.105:0/3248415601 conn(0x7fb0840f2570 msgr2=0x7fb084106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.062+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 shutdown_connections 2024-02-18T01:18:31.066 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:31.062+0000 7fb07affd700 1 -- 172.21.15.105:0/3248415601 wait complete. 2024-02-18T01:18:31.188 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create org_user table v1" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_org_id - v1" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_org_user_org_id_user_id - v1" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:30 smithi155 bash[46955]: t=2024-02-18T01:18:30+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_org_user_user_id - v1" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update org table charset" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update org_user table charset" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate all Read Only Viewers to Viewers" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard table" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard.account_id" 2024-02-18T01:18:31.189 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_account_id_slug" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_tag table" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_tag.dasboard_id_term" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard to dashboard_v1 - v1" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard v2" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_org_id - v2" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_org_id_slug - v2" 2024-02-18T01:18:31.458 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard v1 to v2" 2024-02-18T01:18:31.695 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.log pg_num 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_v1" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard.data to mediumtext v1" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column updated_by in dashboard - v2" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column created_by in dashboard - v2" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column gnetId in dashboard" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for gnetId in dashboard" 2024-02-18T01:18:31.710 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_id in dashboard" 2024-02-18T01:18:31.979 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:18:31.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[20805]: cluster 2024-02-18T01:18:30.283965+0000 mgr.y (mgr.24475) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:31.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[20805]: audit 2024-02-18T01:18:31.058436+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.105:0/3248415601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T01:18:31.980 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[20805]: audit 2024-02-18T01:18:31.538580+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:31.980 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for plugin_id in dashboard" 2024-02-18T01:18:31.980 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_id in dashboard_tag" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard table charset" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_tag table charset" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column folder_id in dashboard" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column isFolder in dashboard" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column has_acl in dashboard" 2024-02-18T01:18:31.981 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:31 smithi155 bash[46955]: t=2024-02-18T01:18:31+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in dashboard" 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[19734]: cluster 2024-02-18T01:18:30.283965+0000 mgr.y (mgr.24475) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[19734]: audit 2024-02-18T01:18:31.058436+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.105:0/3248415601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[19734]: audit 2024-02-18T01:18:31.538580+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[27669]: cluster 2024-02-18T01:18:30.283965+0000 mgr.y (mgr.24475) 50 : cluster [DBG] pgmap v30: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[27669]: audit 2024-02-18T01:18:31.058436+0000 mon.c (mon.2) 25 : audit [DBG] from='client.? v1:172.21.15.105:0/3248415601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".rgw.root", "var": "pg_num"}]: dispatch 2024-02-18T01:18:31.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:31 smithi105 bash[27669]: audit 2024-02-18T01:18:31.538580+0000 mon.a (mon.0) 666 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in dashboard" 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index dashboard_org_id_uid" 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_slug" 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard title length" 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index for dashboard_org_id_title_folder_id" 2024-02-18T01:18:32.244 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning" 2024-02-18T01:18:32.509 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_provisioning v2" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="copy dashboard_provisioning v1 to v2" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="drop dashboard_provisioning_tmp_qwerty" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add check_sum column" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_title" 2024-02-18T01:18:32.510 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="delete tags for deleted dashboards" 2024-02-18T01:18:32.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="delete stars for deleted dashboards" 2024-02-18T01:18:32.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for dashboard_is_folder" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="add index data_source.account_id" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index data_source.account_id_name" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_data_source_account_id - v1" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_data_source_account_id_name - v1" 2024-02-18T01:18:32.813 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table data_source to data_source_v1 - v1" 2024-02-18T01:18:33.082 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create data_source table v2" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_data_source_org_id - v2" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_data_source_org_id_name - v2" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="copy data_source v1 to v2" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table data_source_v1 #2" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:32 smithi155 bash[46955]: t=2024-02-18T01:18:32+0000 lvl=info msg="Executing migration" logger=migrator id="Add column with_credentials" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add secure json data column" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update data_source table charset" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update initial version to 1" 2024-02-18T01:18:33.083 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add read_only data column" 2024-02-18T01:18:33.343 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Migrate logging ds to loki ds" 2024-02-18T01:18:33.343 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update json_data with nulls" 2024-02-18T01:18:33.343 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add uid column" 2024-02-18T01:18:33.344 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid value" 2024-02-18T01:18:33.344 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index datasource_org_id_uid" 2024-02-18T01:18:33.344 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index datasource_org_id_is_default" 2024-02-18T01:18:33.344 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table" 2024-02-18T01:18:33.344 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id" 2024-02-18T01:18:33.613 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.key" 2024-02-18T01:18:33.613 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="add index api_key.account_id_name" 2024-02-18T01:18:33.613 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_api_key_account_id - v1" 2024-02-18T01:18:33.613 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_key - v1" 2024-02-18T01:18:33.613 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_api_key_account_id_name - v1" 2024-02-18T01:18:33.614 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table api_key to api_key_v1 - v1" 2024-02-18T01:18:33.614 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create api_key table v2" 2024-02-18T01:18:33.614 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_api_key_org_id - v2" 2024-02-18T01:18:33.898 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_key - v2" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_api_key_org_id_name - v2" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="copy api_key v1 to v2" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table api_key_v1" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Update api_key table charset" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add expires to api_key table" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="Add service account foreign key" 2024-02-18T01:18:33.899 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v4" 2024-02-18T01:18:34.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[20805]: cluster 2024-02-18T01:18:32.285425+0000 mgr.y (mgr.24475) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:34.168 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="drop table dashboard_snapshot_v4 #1" 2024-02-18T01:18:34.168 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:33 smithi155 bash[46955]: t=2024-02-18T01:18:33+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_snapshot table v5 #2" 2024-02-18T01:18:34.168 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_key - v5" 2024-02-18T01:18:34.169 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-02-18T01:18:34.169 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_dashboard_snapshot_user_id - v5" 2024-02-18T01:18:34.169 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_snapshot to mediumtext v2" 2024-02-18T01:18:34.169 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update dashboard_snapshot table charset" 2024-02-18T01:18:34.169 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external_delete_url to dashboard_snapshots table" 2024-02-18T01:18:34.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:33 smithi105 bash[27669]: cluster 2024-02-18T01:18:32.285425+0000 mgr.y (mgr.24475) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:33 smithi105 bash[19734]: cluster 2024-02-18T01:18:32.285425+0000 mgr.y (mgr.24475) 51 : cluster [DBG] pgmap v31: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add encrypted dashboard json column" 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create quota table v1" 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_quota_org_id_user_id_target - v1" 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update quota table charset" 2024-02-18T01:18:34.456 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create plugin_setting table" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Add column plugin_version to plugin_settings" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update plugin_setting table charset" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create session table" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist table" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old table playlist_item table" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist table v2" 2024-02-18T01:18:34.726 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create playlist item table v2" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist table charset" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update playlist_item table charset" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v2" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="drop preferences table v3" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="create preferences table v3" 2024-02-18T01:18:34.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:34 smithi155 bash[46955]: t=2024-02-18T01:18:34+0000 lvl=info msg="Executing migration" logger=migrator id="Update preferences table charset" 2024-02-18T01:18:35.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[20805]: audit 2024-02-18T01:18:34.183196+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:35.272 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[20805]: cluster 2024-02-18T01:18:34.286109+0000 mgr.y (mgr.24475) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column team_id in preferences" 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Update team_id column values in preferences" 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column week_start in preferences" 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="create alert table v1" 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert org_id & id " 2024-02-18T01:18:35.273 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert state" 2024-02-18T01:18:35.274 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert dashboard_id" 2024-02-18T01:18:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:35 smithi105 bash[27669]: audit 2024-02-18T01:18:34.183196+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:35 smithi105 bash[27669]: cluster 2024-02-18T01:18:34.286109+0000 mgr.y (mgr.24475) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:35 smithi105 bash[19734]: audit 2024-02-18T01:18:34.183196+0000 mon.a (mon.0) 667 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:35 smithi105 bash[19734]: cluster 2024-02-18T01:18:34.286109+0000 mgr.y (mgr.24475) 52 : cluster [DBG] pgmap v32: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v1" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Create alert_rule_tag table v2" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="copy alert_rule_tag v1 to v2" 2024-02-18T01:18:35.575 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop table alert_rule_tag_v1" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification table v1" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column is_default" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column frequency" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column send_reminder" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column disable_resolve_message" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification org_id & name" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert table charset" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert_notification table charset" 2024-02-18T01:18:35.827 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="create notification_journal table v1" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index notification_journal org_id & alert_id & notifier_id" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_notification_journal" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_notification_state table v1" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add for to alert table" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:35 smithi155 bash[46955]: t=2024-02-18T01:18:35+0000 lvl=info msg="Executing migration" logger=migrator id="Add column uid in alert_notification" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Update uid column values in alert_notification" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index alert_notification_org_id_uid" 2024-02-18T01:18:36.112 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index org_id_name" 2024-02-18T01:18:36.274 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add column secure_settings in alert_notification" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert.settings to mediumtext" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_notification_state_alert_id" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add non-unique index alert_rule_tag_alert_id" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Drop old annotation table v4" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="create annotation table v5" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 0 v3" 2024-02-18T01:18:36.382 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 1 v3" 2024-02-18T01:18:36.383 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 2 v3" 2024-02-18T01:18:36.684 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 3 v3" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="add index annotation 4 v3" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Update annotation table charset" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add column region_id to annotation table" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Drop category_id index" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add column tags to annotation table" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v2" 2024-02-18T01:18:36.685 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:35.523368+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.685 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:35.525483+0000 mon.a (mon.0) 669 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:36.685 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:35.527506+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:36.685 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:35.539066+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.685 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: cephadm 2024-02-18T01:18:35.547110+0000 mgr.y (mgr.24475) 53 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T01:18:36.686 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: cephadm 2024-02-18T01:18:35.552199+0000 mgr.y (mgr.24475) 54 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi105 2024-02-18T01:18:36.686 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:36.272655+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.686 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:36.273424+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-02-18T01:18:36.686 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[20805]: audit 2024-02-18T01:18:36.274113+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add unique index annotation_tag.annotation_id_tag_id" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Create annotation_tag table v3" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="copy annotation_tag v2 to v3" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="drop table annotation_tag_v2" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Update alert annotations and set TEXT to empty" 2024-02-18T01:18:36.954 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add created time to annotation table" 2024-02-18T01:18:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:35.523368+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:35.525483+0000 mon.a (mon.0) 669 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:35.527506+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:35.539066+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: cephadm 2024-02-18T01:18:35.547110+0000 mgr.y (mgr.24475) 53 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: cephadm 2024-02-18T01:18:35.552199+0000 mgr.y (mgr.24475) 54 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi105 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:36.272655+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:36.273424+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[27669]: audit 2024-02-18T01:18:36.274113+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:35.523368+0000 mon.a (mon.0) 668 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:35.525483+0000 mon.a (mon.0) 669 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:35.527506+0000 mon.a (mon.0) 670 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:35.539066+0000 mon.a (mon.0) 671 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: cephadm 2024-02-18T01:18:35.547110+0000 mgr.y (mgr.24475) 53 : cephadm [INF] Reconfiguring alertmanager.a (dependencies changed)... 2024-02-18T01:18:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: cephadm 2024-02-18T01:18:35.552199+0000 mgr.y (mgr.24475) 54 : cephadm [INF] Reconfiguring daemon alertmanager.a on smithi105 2024-02-18T01:18:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:36.272655+0000 mon.a (mon.0) 672 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:36.273424+0000 mon.a (mon.0) 673 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]: dispatch 2024-02-18T01:18:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[19734]: audit 2024-02-18T01:18:36.274113+0000 mon.a (mon.0) 674 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]: dispatch 2024-02-18T01:18:36.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:36] "GET /metrics HTTP/1.1" 200 172138 "" "Prometheus/2.33.4" 2024-02-18T01:18:37.239 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add updated time to annotation table" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:36 smithi155 bash[46955]: t=2024-02-18T01:18:36+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for created in annotation table" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for updated in annotation table" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Convert existing annotations from seconds to milliseconds" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add epoch_end column" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for epoch_end" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Make epoch_end the same as epoch" 2024-02-18T01:18:37.240 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Move region to single row" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch from annotation table" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for org_id_epoch_end_epoch on annotation table" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Remove index org_id_epoch_epoch_end from annotation table" 2024-02-18T01:18:37.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add index for alert_id on annotation table" 2024-02-18T01:18:37.495 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="create test_data table" 2024-02-18T01:18:37.495 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard_version table v1" 2024-02-18T01:18:37.495 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_version.dashboard_id" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Set dashboard version to 1 where 0" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="save existing dashboard data in dashboard_version table v1" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="alter dashboard_version.data to mediumtext v1" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="create team table" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index team.org_id" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_org_id_name" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="create team member table" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.org_id" 2024-02-18T01:18:37.812 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: cluster 2024-02-18T01:18:36.286600+0000 mgr.y (mgr.24475) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: audit 2024-02-18T01:18:36.547836+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: audit 2024-02-18T01:18:36.548018+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: audit 2024-02-18T01:18:36.548172+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: cluster 2024-02-18T01:18:36.548546+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: audit 2024-02-18T01:18:36.550665+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:18:37.813 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[20805]: audit 2024-02-18T01:18:36.551889+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: cluster 2024-02-18T01:18:36.286600+0000 mgr.y (mgr.24475) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: audit 2024-02-18T01:18:36.547836+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: audit 2024-02-18T01:18:36.548018+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: audit 2024-02-18T01:18:36.548172+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: cluster 2024-02-18T01:18:36.548546+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: audit 2024-02-18T01:18:36.550665+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[27669]: audit 2024-02-18T01:18:36.551889+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: cluster 2024-02-18T01:18:36.286600+0000 mgr.y (mgr.24475) 55 : cluster [DBG] pgmap v33: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: audit 2024-02-18T01:18:36.547836+0000 mon.a (mon.0) 675 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-02-18T01:18:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: audit 2024-02-18T01:18:36.548018+0000 mon.a (mon.0) 676 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.6", "id": [1, 5]}]': finished 2024-02-18T01:18:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: audit 2024-02-18T01:18:36.548172+0000 mon.a (mon.0) 677 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [1, 2]}]': finished 2024-02-18T01:18:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: cluster 2024-02-18T01:18:36.548546+0000 mon.a (mon.0) 678 : cluster [DBG] osdmap e63: 8 total, 8 up, 8 in 2024-02-18T01:18:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: audit 2024-02-18T01:18:36.550665+0000 mon.a (mon.0) 679 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-02-18T01:18:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:37 smithi105 bash[19734]: audit 2024-02-18T01:18:36.551889+0000 mon.a (mon.0) 680 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2024-02-18T01:18:38.064 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_member_org_id_team_id_user_id" 2024-02-18T01:18:38.064 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_member.team_id" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add column email to team table" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add column external to team_member table" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="Add column permission to team_member table" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:37 smithi155 bash[46955]: t=2024-02-18T01:18:37+0000 lvl=info msg="Executing migration" logger=migrator id="create dashboard acl table" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_dashboard_id" 2024-02-18T01:18:38.065 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_user_id" 2024-02-18T01:18:38.066 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index dashboard_acl_dashboard_id_team_id" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_user_id" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_team_id" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_acl_org_id_role" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index dashboard_permission" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="save default acl rules in dashboard_acl table" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="delete acl rules for deleted dashboards and folders" 2024-02-18T01:18:38.358 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create tag table" 2024-02-18T01:18:38.626 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[20805]: audit 2024-02-18T01:18:37.305413+0000 mgr.y (mgr.24475) 56 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:38.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[20805]: audit 2024-02-18T01:18:37.550163+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:18:38.627 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[20805]: cluster 2024-02-18T01:18:37.550232+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index tag.key_value" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create login attempt table" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="add index login_attempt.username" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="drop index IDX_login_attempt_username - v1" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create login_attempt v2" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_login_attempt_username - v2" 2024-02-18T01:18:38.627 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="copy login_attempt v1 to v2" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="drop login_attempt_tmp_qwerty" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth table" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="alter user_auth.auth_id to length 190" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth access token to user_auth" 2024-02-18T01:18:38.929 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth refresh token to user_auth" 2024-02-18T01:18:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[19734]: audit 2024-02-18T01:18:37.305413+0000 mgr.y (mgr.24475) 56 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[19734]: audit 2024-02-18T01:18:37.550163+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:18:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[19734]: cluster 2024-02-18T01:18:37.550232+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T01:18:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[27669]: audit 2024-02-18T01:18:37.305413+0000 mgr.y (mgr.24475) 56 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[27669]: audit 2024-02-18T01:18:37.550163+0000 mon.a (mon.0) 681 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2024-02-18T01:18:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:38 smithi105 bash[27669]: cluster 2024-02-18T01:18:37.550232+0000 mon.a (mon.0) 682 : cluster [DBG] osdmap e64: 8 total, 8 up, 8 in 2024-02-18T01:18:39.206 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth token type to user_auth" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Add OAuth expiry to user_auth" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:38 smithi155 bash[46955]: t=2024-02-18T01:18:38+0000 lvl=info msg="Executing migration" logger=migrator id="Add index to user_id column in user_auth" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="create server_lock table" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index server_lock.operation_uid" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="create user auth token table" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.auth_token" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_auth_token.prev_auth_token" 2024-02-18T01:18:39.207 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_auth_token.user_id" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="Add revoked_at to the user auth token" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="create cache_data table" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index cache_data.cache_key" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="create short_url table v1" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index short_url.org_id-uid" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition table" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition table" 2024-02-18T01:18:39.472 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and title columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[20805]: cluster 2024-02-18T01:18:38.287429+0000 mgr.y (mgr.24475) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:39.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[20805]: cluster 2024-02-18T01:18:38.556825+0000 mon.a (mon.0) 683 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition on org_id and uid columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition table data column to mediumtext in mysql" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and title columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="drop index in alert_definition on org_id and uid columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and title columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index in alert_definition on org_id and uid columns" 2024-02-18T01:18:39.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="Add column paused in alert_definition" 2024-02-18T01:18:39.940 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:39 smithi105 bash[19734]: cluster 2024-02-18T01:18:38.287429+0000 mgr.y (mgr.24475) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:39.941 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:39 smithi105 bash[19734]: cluster 2024-02-18T01:18:38.556825+0000 mon.a (mon.0) 683 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-18T01:18:39.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:39 smithi105 bash[27669]: cluster 2024-02-18T01:18:38.287429+0000 mgr.y (mgr.24475) 57 : cluster [DBG] pgmap v36: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:39.941 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:39 smithi105 bash[27669]: cluster 2024-02-18T01:18:38.556825+0000 mon.a (mon.0) 683 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition table" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="delete alert_definition_version table" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="recreate alert_definition_version table" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_definition_version table data column to mediumtext in mysql" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="drop alert_definition_version table" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:39 smithi155 bash[46955]: t=2024-02-18T01:18:39+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_instance table" 2024-02-18T01:18:40.031 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-02-18T01:18:40.291 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_instance table on def_org_id, current_state columns" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column current_state_end to alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="remove index def_org_id, current_state on alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_org_id to rule_org_id in alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="rename def_uid to rule_uid in alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index rule_org_id, current_state on alert_instance" 2024-02-18T01:18:40.292 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule table" 2024-02-18T01:18:40.293 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and title columns" 2024-02-18T01:18:40.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- 172.21.15.105:0/2268301803 >> v1:172.21.15.105:6789/0 conn(0x7f47fc106640 legacy=0x7f47fc108a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:40.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- 172.21.15.105:0/2268301803 shutdown_connections 2024-02-18T01:18:40.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- 172.21.15.105:0/2268301803 >> 172.21.15.105:0/2268301803 conn(0x7f47fc0f3690 msgr2=0x7f47fc0f5ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:40.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- 172.21.15.105:0/2268301803 shutdown_connections 2024-02-18T01:18:40.340 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- 172.21.15.105:0/2268301803 wait complete. 2024-02-18T01:18:40.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 Processor -- start 2024-02-18T01:18:40.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.334+0000 7f4800d17700 1 -- start start 2024-02-18T01:18:40.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47fc119680 con 0x7f47fc106640 2024-02-18T01:18:40.341 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47fc11a860 con 0x7f47fc06b0f0 2024-02-18T01:18:40.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47fc11ba60 con 0x7f47fc06bbe0 2024-02-18T01:18:40.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47fad9d700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f47fc106640 0x7f47fc117d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:47580/0 (socket says 172.21.15.105:47580) 2024-02-18T01:18:40.342 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47fad9d700 1 -- 172.21.15.105:0/815466649 learned_addr learned my addr 172.21.15.105:0/815466649 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:40.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2522608438 0 0) 0x7f47fc11ba60 con 0x7f47fc06bbe0 2024-02-18T01:18:40.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47e80033f0 con 0x7f47fc06bbe0 2024-02-18T01:18:40.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1122779925 0 0) 0x7f47fc119680 con 0x7f47fc106640 2024-02-18T01:18:40.343 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47fc11ba60 con 0x7f47fc106640 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3762388282 0 0) 0x7f47fc11a860 con 0x7f47fc06b0f0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47fc119680 con 0x7f47fc06b0f0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 332552362 0 0) 0x7f47fc11ba60 con 0x7f47fc106640 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47fc11a860 con 0x7f47fc106640 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2961898322 0 0) 0x7f47e80033f0 con 0x7f47fc06bbe0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47fc11ba60 con 0x7f47fc06bbe0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f47f4002c60 con 0x7f47fc106640 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f47e40018c0 con 0x7f47fc06bbe0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 121949146 0 0) 0x7f47fc119680 con 0x7f47fc06b0f0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47e80033f0 con 0x7f47fc06b0f0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f47ec001c40 con 0x7f47fc06b0f0 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4023213419 0 0) 0x7f47fc11a860 con 0x7f47fc106640 2024-02-18T01:18:40.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 >> v1:172.21.15.105:6790/0 conn(0x7f47fc06bbe0 legacy=0x7f47fc103310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 >> v1:172.21.15.155:6789/0 conn(0x7f47fc06b0f0 legacy=0x7f47fc102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47fc11cc60 con 0x7f47fc106640 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47fc11bc50 con 0x7f47fc106640 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f47fc11c0c0 con 0x7f47fc106640 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f47f4001bb0 con 0x7f47fc106640 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f47f40037a0 con 0x7f47fc106640 2024-02-18T01:18:40.345 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47dc004f20 con 0x7f47fc106640 2024-02-18T01:18:40.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.338+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f47f4003900 con 0x7f47fc106640 2024-02-18T01:18:40.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.342+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f47f4078810 con 0x7f47fc106640 2024-02-18T01:18:40.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.346+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f47f404c4d0 con 0x7f47fc106640 2024-02-18T01:18:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.530+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"} v 0) v1 -- 0x7f47dc005750 con 0x7f47fc106640 2024-02-18T01:18:40.535 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.530+0000 7f47f3fff700 1 -- 172.21.15.105:0/815466649 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]=0 v64) v1 ==== 104+0+11 (unknown 2323790917 0 3519293104) 0x7f47f404a940 con 0x7f47fc106640 2024-02-18T01:18:40.535 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-02-18T01:18:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 >> v1:172.21.15.105:6800/2148176012 conn(0x7f47e80622b0 legacy=0x7f47e8064770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:40.538 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 >> v1:172.21.15.105:6789/0 conn(0x7f47fc106640 legacy=0x7f47fc117d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 shutdown_connections 2024-02-18T01:18:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 >> 172.21.15.105:0/815466649 conn(0x7f47fc0f3690 msgr2=0x7f47fc108a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 shutdown_connections 2024-02-18T01:18:40.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:40.534+0000 7f4800d17700 1 -- 172.21.15.105:0/815466649 wait complete. 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id and uid columns" 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule table data column to mediumtext in mysql" 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule" 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule" 2024-02-18T01:18:40.577 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule" 2024-02-18T01:18:40.829 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[20805]: audit 2024-02-18T01:18:40.534704+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? v1:172.21.15.105:0/815466649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T01:18:40.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="remove unique index from alert_rule on org_id, title columns" 2024-02-18T01:18:40.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-02-18T01:18:40.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add dashboard_uid column to alert_rule" 2024-02-18T01:18:40.829 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add panel_id column to alert_rule" 2024-02-18T01:18:40.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-02-18T01:18:40.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="create alert_rule_version table" 2024-02-18T01:18:40.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-02-18T01:18:40.830 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-02-18T01:18:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:40 smithi105 bash[19734]: audit 2024-02-18T01:18:40.534704+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? v1:172.21.15.105:0/815466649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T01:18:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:40 smithi105 bash[27669]: audit 2024-02-18T01:18:40.534704+0000 mon.a (mon.0) 684 : audit [DBG] from='client.? v1:172.21.15.105:0/815466649' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.log", "var": "pg_num"}]: dispatch 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="alter alert_rule_version table data column to mediumtext in mysql" 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column for to alert_rule_version" 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column annotations to alert_rule_version" 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="add column labels to alert_rule_version" 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id=create_alert_configuration_table 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:40 smithi155 bash[46955]: t=2024-02-18T01:18:40+0000 lvl=info msg="Executing migration" logger=migrator id="Add column default in alert_configuration" 2024-02-18T01:18:41.132 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-02-18T01:18:41.133 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add column org_id in alert_configuration" 2024-02-18T01:18:41.133 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index in alert_configuration table on org_id column" 2024-02-18T01:18:41.409 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id=create_ngalert_configuration_table 2024-02-18T01:18:41.409 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index in ngalert_configuration on org_id column" 2024-02-18T01:18:41.409 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="clear migration entry \"remove unified alerting data\"" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="move dashboard alerts to unified alerting" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element table v1" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element org_id-folder_id-name-kind" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create library_element_connection table v1" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index library_element_connection element_id-kind-connection_id" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index library_element org_id_uid" 2024-02-18T01:18:41.410 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="clone move dashboard alerts to unified alerting" 2024-02-18T01:18:41.432 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:41 smithi105 systemd[1]: Stopping Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:18:41.433 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:41 smithi105 bash[53881]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-alertmanager.a 2024-02-18T01:18:41.433 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:41 smithi105 bash[52148]: level=info ts=2024-02-18T01:18:41.354Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-18T01:18:41.456 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.control pg_num 2024-02-18T01:18:41.670 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[20805]: cluster 2024-02-18T01:18:40.288001+0000 mgr.y (mgr.24475) 58 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create data_keys table" 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create kv_store table v1" 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index kv_store.org_id-namespace-key" 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="update dashboard_uid and panel_id from existing annotations" 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create permission table" 2024-02-18T01:18:41.670 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index permission.role_id" 2024-02-18T01:18:41.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:41 smithi105 bash[19734]: cluster 2024-02-18T01:18:40.288001+0000 mgr.y (mgr.24475) 58 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:41.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:41 smithi105 bash[27669]: cluster 2024-02-18T01:18:40.288001+0000 mgr.y (mgr.24475) 58 : cluster [DBG] pgmap v37: 132 pgs: 3 peering, 129 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:18:41.947 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_id_action_scope" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create role table" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add column display_name" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add column group_name" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index role.org_id" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role_org_id_name" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index role_org_id_uid" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="create team role table" 2024-02-18T01:18:41.948 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.org_id" 2024-02-18T01:18:42.199 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index team_role_org_id_team_id_role_id" 2024-02-18T01:18:42.199 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:41 smithi155 bash[46955]: t=2024-02-18T01:18:41+0000 lvl=info msg="Executing migration" logger=migrator id="add index team_role.team_id" 2024-02-18T01:18:42.199 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="create user role table" 2024-02-18T01:18:42.200 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.org_id" 2024-02-18T01:18:42.200 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index user_role_org_id_user_id_role_id" 2024-02-18T01:18:42.200 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add index user_role.user_id" 2024-02-18T01:18:42.200 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="create builtin role table" 2024-02-18T01:18:42.200 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.role_id" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.name" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="Add column org_id to builtin_role table" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add index builtin_role.org_id" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_org_id_role_id_role" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="Remove unique index role_org_id_uid" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index role.uid" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="create seed assignment table" 2024-02-18T01:18:42.469 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Executing migration" logger=migrator id="add unique index builtin_role_role_name" 2024-02-18T01:18:42.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:42 smithi105 bash[53881]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-alertmanager-a 2024-02-18T01:18:42.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:42 smithi105 systemd[1]: Stopped Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:42.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:42 smithi105 systemd[1]: Started Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:42.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="migrations completed" logger=migrator performed=381 skipped=0 duration=13.096275506s 2024-02-18T01:18:42.744 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:42 smithi155 bash[46955]: t=2024-02-18T01:18:42+0000 lvl=info msg="Created default organization" logger=sqlstore 2024-02-18T01:18:43.375 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.316Z caller=main.go:225 msg="Starting Alertmanager" version="(version=0.23.0, branch=HEAD, revision=61046b17771a57cfd4c4a51be370ab930a4d7d54)" 2024-02-18T01:18:43.375 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.316Z caller=main.go:226 build_context="(go=go1.16.7, user=root@e21a959be8d2, date=20210825-10:48:55)" 2024-02-18T01:18:43.375 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.319Z caller=cluster.go:184 component=cluster msg="setting advertise address explicitly" addr=172.21.15.105 port=9094 2024-02-18T01:18:43.375 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.321Z caller=cluster.go:671 component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[19734]: audit 2024-02-18T01:18:42.201066+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[19734]: cephadm 2024-02-18T01:18:42.203035+0000 mgr.y (mgr.24475) 59 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[19734]: cephadm 2024-02-18T01:18:42.209189+0000 mgr.y (mgr.24475) 60 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi155 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[19734]: cluster 2024-02-18T01:18:42.289363+0000 mgr.y (mgr.24475) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[27669]: audit 2024-02-18T01:18:42.201066+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[27669]: cephadm 2024-02-18T01:18:42.203035+0000 mgr.y (mgr.24475) 59 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-18T01:18:43.376 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[27669]: cephadm 2024-02-18T01:18:42.209189+0000 mgr.y (mgr.24475) 60 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi155 2024-02-18T01:18:43.377 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[27669]: cluster 2024-02-18T01:18:42.289363+0000 mgr.y (mgr.24475) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:43.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[20805]: audit 2024-02-18T01:18:42.201066+0000 mon.a (mon.0) 685 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:43.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[20805]: cephadm 2024-02-18T01:18:42.203035+0000 mgr.y (mgr.24475) 59 : cephadm [INF] Reconfiguring prometheus.a (dependencies changed)... 2024-02-18T01:18:43.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[20805]: cephadm 2024-02-18T01:18:42.209189+0000 mgr.y (mgr.24475) 60 : cephadm [INF] Reconfiguring daemon prometheus.a on smithi155 2024-02-18T01:18:43.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[20805]: cluster 2024-02-18T01:18:42.289363+0000 mgr.y (mgr.24475) 61 : cluster [DBG] pgmap v38: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:43.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="Initialising plugins" logger=plugin.manager 2024-02-18T01:18:43.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=input 2024-02-18T01:18:43.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=grafana-piechart-panel 2024-02-18T01:18:43.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="Plugin registered" logger=plugin.manager pluginId=vonage-status-panel 2024-02-18T01:18:43.494 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="Live Push Gateway initialization" logger=live.push_http 2024-02-18T01:18:43.495 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" logger=provisioning.datasources filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-02-18T01:18:43.495 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="inserting datasource from configuration " logger=provisioning.datasources name=Dashboard1 uid=P43CA22E17D0F9596 2024-02-18T01:18:43.739 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.374Z caller=coordinator.go:113 component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T01:18:43.740 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.375Z caller=coordinator.go:126 component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-02-18T01:18:43.740 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.379Z caller=main.go:518 msg=Listening address=:9093 2024-02-18T01:18:43.740 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:43 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:43.379Z caller=tls_config.go:191 msg="TLS is disabled." http2=false 2024-02-18T01:18:43.993 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="warming cache for startup" logger=ngalert 2024-02-18T01:18:43.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="HTTP Server Listen" logger=http.server address=[::]:3000 protocol=https subUrl= socket= 2024-02-18T01:18:43.994 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:18:43 smithi155 bash[46955]: t=2024-02-18T01:18:43+0000 lvl=info msg="starting MultiOrg Alertmanager" logger=ngalert.multiorg.alertmanager 2024-02-18T01:18:44.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:44 smithi155 bash[20805]: cluster 2024-02-18T01:18:43.197488+0000 mon.a (mon.0) 686 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-18T01:18:44.408 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:44 smithi155 bash[20805]: cluster 2024-02-18T01:18:43.197543+0000 mon.a (mon.0) 687 : cluster [INF] Cluster is now healthy 2024-02-18T01:18:44.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:44 smithi105 bash[19734]: cluster 2024-02-18T01:18:43.197488+0000 mon.a (mon.0) 686 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-18T01:18:44.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:44 smithi105 bash[19734]: cluster 2024-02-18T01:18:43.197543+0000 mon.a (mon.0) 687 : cluster [INF] Cluster is now healthy 2024-02-18T01:18:44.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:44 smithi105 bash[27669]: cluster 2024-02-18T01:18:43.197488+0000 mon.a (mon.0) 686 : cluster [INF] Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-02-18T01:18:44.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:44 smithi105 bash[27669]: cluster 2024-02-18T01:18:43.197543+0000 mon.a (mon.0) 687 : cluster [INF] Cluster is now healthy 2024-02-18T01:18:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:45 smithi105 bash[19734]: cluster 2024-02-18T01:18:44.290043+0000 mgr.y (mgr.24475) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:45 smithi105 bash[27669]: cluster 2024-02-18T01:18:44.290043+0000 mgr.y (mgr.24475) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:45.490 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:45 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:45.322Z caller=cluster.go:696 component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000827502s 2024-02-18T01:18:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:45 smithi155 bash[20805]: cluster 2024-02-18T01:18:44.290043+0000 mgr.y (mgr.24475) 62 : cluster [DBG] pgmap v39: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 2 objects/s recovering 2024-02-18T01:18:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:46] "GET /metrics HTTP/1.1" 200 172138 "" "Prometheus/2.33.4" 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:46 smithi155 systemd[1]: Stopping Ceph prometheus.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:46 smithi155 bash[47691]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-prometheus.a 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-18T01:18:47.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.119Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.120Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.120Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.120Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-18T01:18:47.245 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[46533]: ts=2024-02-18T01:18:47.120Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-18T01:18:47.286 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:47.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[19734]: cluster 2024-02-18T01:18:46.290741+0000 mgr.y (mgr.24475) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-02-18T01:18:47.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[27669]: cluster 2024-02-18T01:18:46.290741+0000 mgr.y (mgr.24475) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-02-18T01:18:47.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[20805]: cluster 2024-02-18T01:18:46.290741+0000 mgr.y (mgr.24475) 63 : cluster [DBG] pgmap v40: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-02-18T01:18:47.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:48.167 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[20046]: [18/Feb/2024:01:18:47] ENGINE Bus STOPPING 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[20046]: [18/Feb/2024:01:18:47] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[20046]: [18/Feb/2024:01:18:47] ENGINE Bus STOPPED 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:47 smithi105 bash[20046]: [18/Feb/2024:01:18:47] ENGINE Bus STARTING 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Serving on http://:::9283 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STARTED 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STOPPING 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STOPPED 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STARTING 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Serving on http://:::9283 2024-02-18T01:18:48.168 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STARTED 2024-02-18T01:18:48.243 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 bash[47691]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-prometheus-a 2024-02-18T01:18:48.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 systemd[1]: Stopped Ceph prometheus.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:48.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:47 smithi155 systemd[1]: Started Ceph prometheus.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STOPPING 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STOPPED 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STARTING 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Serving on http://:::9283 2024-02-18T01:18:48.454 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[20046]: [18/Feb/2024:01:18:48] ENGINE Bus STARTED 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.316642+0000 mgr.y (mgr.24475) 64 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.886896+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.893988+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.894674+0000 mgr.y (mgr.24475) 65 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.896593+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.897099+0000 mgr.y (mgr.24475) 66 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.905974+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.916714+0000 mon.a (mon.0) 692 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.917399+0000 mgr.y (mgr.24475) 67 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.923601+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.924270+0000 mgr.y (mgr.24475) 68 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.932547+0000 mon.a (mon.0) 694 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.941466+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.949490+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.958860+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[27669]: audit 2024-02-18T01:18:47.965986+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.316642+0000 mgr.y (mgr.24475) 64 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.886896+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.893988+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.894674+0000 mgr.y (mgr.24475) 65 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.896593+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.897099+0000 mgr.y (mgr.24475) 66 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.905974+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.916714+0000 mon.a (mon.0) 692 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.917399+0000 mgr.y (mgr.24475) 67 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.923601+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.924270+0000 mgr.y (mgr.24475) 68 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.932547+0000 mon.a (mon.0) 694 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.941466+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.949490+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.958860+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:48 smithi105 bash[19734]: audit 2024-02-18T01:18:47.965986+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:49.243 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.016Z caller=main.go:512 level=info msg="Starting Prometheus" version="(version=2.33.4, branch=HEAD, revision=83032011a5d3e6102624fe58241a374a7201fee8)" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.016Z caller=main.go:517 level=info build_context="(go=go1.17.7, user=root@d13bf69e7be8, date=20220222-16:51:28)" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.017Z caller=main.go:518 level=info host_details="(Linux 4.15.0-213-generic #224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023 x86_64 smithi155 (none))" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.017Z caller=main.go:519 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.017Z caller=main.go:520 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.020Z caller=web.go:570 level=info component=web msg="Start listening for connections" address=:9095 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.020Z caller=main.go:923 level=info msg="Starting TSDB ..." 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.021Z caller=tls_config.go:195 level=info component=web msg="TLS is disabled." http2=false 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.023Z caller=head.go:493 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.023Z caller=head.go:527 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.022µs 2024-02-18T01:18:49.244 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.023Z caller=head.go:533 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-02-18T01:18:49.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.316642+0000 mgr.y (mgr.24475) 64 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.886896+0000 mon.a (mon.0) 688 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.893988+0000 mon.a (mon.0) 689 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.894674+0000 mgr.y (mgr.24475) 65 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.896593+0000 mon.a (mon.0) 690 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.897099+0000 mgr.y (mgr.24475) 66 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi105.front.sepia.ceph.com:9093"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.905974+0000 mon.a (mon.0) 691 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.916714+0000 mon.a (mon.0) 692 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.917399+0000 mgr.y (mgr.24475) 67 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.923601+0000 mon.a (mon.0) 693 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.924270+0000 mgr.y (mgr.24475) 68 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi155.front.sepia.ceph.com:9095"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.932547+0000 mon.a (mon.0) 694 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.941466+0000 mon.a (mon.0) 695 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.949490+0000 mon.a (mon.0) 696 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.958860+0000 mon.a (mon.0) 697 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:49.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:48 smithi155 bash[20805]: audit 2024-02-18T01:18:47.965986+0000 mon.a (mon.0) 698 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.305Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.306Z caller=head.go:604 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.306Z caller=head.go:610 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=32.861µs wal_replay_duration=282.145614ms total_replay_duration=282.190255ms 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.311Z caller=main.go:944 level=info fs_type=XFS_SUPER_MAGIC 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.312Z caller=main.go:947 level=info msg="TSDB started" 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.312Z caller=main.go:1128 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.343Z caller=main.go:1165 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=31.427592ms db_storage=1.4µs remote_storage=2.813µs web_handler=803ns query_engine=1.274µs scrape=636.381µs scrape_sd=86.749µs notify=46.502µs notify_sd=53.179µs rules=29.911422ms 2024-02-18T01:18:49.744 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[47821]: ts=2024-02-18T01:18:49.343Z caller=main.go:896 level=info msg="Server is ready to receive web requests." 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[19734]: audit 2024-02-18T01:18:47.942152+0000 mgr.y (mgr.24475) 69 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[19734]: audit 2024-02-18T01:18:47.950082+0000 mgr.y (mgr.24475) 70 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[19734]: cluster 2024-02-18T01:18:48.292188+0000 mgr.y (mgr.24475) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s; 10 B/s, 1 objects/s recovering 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[27669]: audit 2024-02-18T01:18:47.942152+0000 mgr.y (mgr.24475) 69 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[27669]: audit 2024-02-18T01:18:47.950082+0000 mgr.y (mgr.24475) 70 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:50.156 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:49 smithi105 bash[27669]: cluster 2024-02-18T01:18:48.292188+0000 mgr.y (mgr.24475) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s; 10 B/s, 1 objects/s recovering 2024-02-18T01:18:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[20805]: audit 2024-02-18T01:18:47.942152+0000 mgr.y (mgr.24475) 69 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-02-18T01:18:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[20805]: audit 2024-02-18T01:18:47.950082+0000 mgr.y (mgr.24475) 70 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi155.front.sepia.ceph.com:3000"}]: dispatch 2024-02-18T01:18:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:49 smithi155 bash[20805]: cluster 2024-02-18T01:18:48.292188+0000 mgr.y (mgr.24475) 71 : cluster [DBG] pgmap v41: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 0 B/s wr, 4 op/s; 10 B/s, 1 objects/s recovering 2024-02-18T01:18:50.367 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- 172.21.15.105:0/1478612491 >> v1:172.21.15.105:6789/0 conn(0x7ff620106ec0 legacy=0x7ff6201092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- 172.21.15.105:0/1478612491 shutdown_connections 2024-02-18T01:18:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- 172.21.15.105:0/1478612491 >> 172.21.15.105:0/1478612491 conn(0x7ff62006d900 msgr2=0x7ff62006fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- 172.21.15.105:0/1478612491 shutdown_connections 2024-02-18T01:18:50.368 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- 172.21.15.105:0/1478612491 wait complete. 2024-02-18T01:18:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 Processor -- start 2024-02-18T01:18:50.369 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.362+0000 7ff625575700 1 -- start start 2024-02-18T01:18:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff62011db10 con 0x7ff620106ec0 2024-02-18T01:18:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff62011ed10 con 0x7ff620104070 2024-02-18T01:18:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7ff62011ff10 con 0x7ff62010a9d0 2024-02-18T01:18:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff61f7fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7ff62010a9d0 0x7ff62011c1d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:57976/0 (socket says 172.21.15.105:57976) 2024-02-18T01:18:50.370 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff61f7fe700 1 -- 172.21.15.105:0/2999690424 learned_addr learned my addr 172.21.15.105:0/2999690424 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647326503 0 0) 0x7ff62011ff10 con 0x7ff62010a9d0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c0033f0 con 0x7ff62010a9d0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555808562 0 0) 0x7ff62011db10 con 0x7ff620106ec0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62011ff10 con 0x7ff620106ec0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1925985225 0 0) 0x7ff62011ed10 con 0x7ff620104070 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff62011db10 con 0x7ff620104070 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 364991279 0 0) 0x7ff60c0033f0 con 0x7ff62010a9d0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff62011ed10 con 0x7ff62010a9d0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff610001c80 con 0x7ff62010a9d0 2024-02-18T01:18:50.372 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 864298583 0 0) 0x7ff62011ff10 con 0x7ff620106ec0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7ff60c0033f0 con 0x7ff620106ec0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff608001f90 con 0x7ff620106ec0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3974068074 0 0) 0x7ff62011ed10 con 0x7ff62010a9d0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 >> v1:172.21.15.155:6789/0 conn(0x7ff620104070 legacy=0x7ff620100400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 >> v1:172.21.15.105:6789/0 conn(0x7ff620106ec0 legacy=0x7ff620100b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff620121110 con 0x7ff62010a9d0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff62011dd00 con 0x7ff62010a9d0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7ff62011e170 con 0x7ff62010a9d0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7ff6100027a0 con 0x7ff62010a9d0 2024-02-18T01:18:50.373 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7ff610005da0 con 0x7ff62010a9d0 2024-02-18T01:18:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff600004f20 con 0x7ff62010a9d0 2024-02-18T01:18:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.366+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7ff610006ff0 con 0x7ff62010a9d0 2024-02-18T01:18:50.374 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.370+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7ff610078620 con 0x7ff62010a9d0 2024-02-18T01:18:50.379 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.374+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7ff61004b520 con 0x7ff62010a9d0 2024-02-18T01:18:50.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.554+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"} v 0) v1 -- 0x7ff600005750 con 0x7ff62010a9d0 2024-02-18T01:18:50.560 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.554+0000 7ff617fff700 1 -- 172.21.15.105:0/2999690424 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]=0 v64) v1 ==== 108+0+11 (unknown 2613393816 0 3519293104) 0x7ff61004b520 con 0x7ff62010a9d0 2024-02-18T01:18:50.561 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-02-18T01:18:50.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 >> v1:172.21.15.105:6800/2148176012 conn(0x7ff60c062020 legacy=0x7ff60c0644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:50.564 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 >> v1:172.21.15.105:6790/0 conn(0x7ff62010a9d0 legacy=0x7ff62011c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:50.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 shutdown_connections 2024-02-18T01:18:50.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 >> 172.21.15.105:0/2999690424 conn(0x7ff62006d900 msgr2=0x7ff620104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:50.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 shutdown_connections 2024-02-18T01:18:50.565 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:50.558+0000 7ff625575700 1 -- 172.21.15.105:0/2999690424 wait complete. 2024-02-18T01:18:51.221 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:50 smithi105 bash[19734]: audit 2024-02-18T01:18:50.560157+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? v1:172.21.15.105:0/2999690424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T01:18:51.221 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:50 smithi105 bash[27669]: audit 2024-02-18T01:18:50.560157+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? v1:172.21.15.105:0/2999690424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T01:18:51.222 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd pool get default.rgw.meta pg_num 2024-02-18T01:18:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:50 smithi155 bash[20805]: audit 2024-02-18T01:18:50.560157+0000 mon.c (mon.2) 26 : audit [DBG] from='client.? v1:172.21.15.105:0/2999690424' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.control", "var": "pg_num"}]: dispatch 2024-02-18T01:18:52.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:51 smithi105 bash[27669]: cluster 2024-02-18T01:18:50.292856+0000 mgr.y (mgr.24475) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:51 smithi105 bash[19734]: cluster 2024-02-18T01:18:50.292856+0000 mgr.y (mgr.24475) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:52.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:51 smithi155 bash[20805]: cluster 2024-02-18T01:18:50.292856+0000 mgr.y (mgr.24475) 72 : cluster [DBG] pgmap v42: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:53.739 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:18:53 smithi105 bash[54021]: level=info ts=2024-02-18T01:18:53.325Z caller=cluster.go:688 component=cluster msg="gossip settled; proceeding" elapsed=10.003415953s 2024-02-18T01:18:54.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:53 smithi105 bash[27669]: cluster 2024-02-18T01:18:52.294185+0000 mgr.y (mgr.24475) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:54.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:53 smithi105 bash[27669]: audit 2024-02-18T01:18:53.792181+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:54.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:53 smithi105 bash[19734]: cluster 2024-02-18T01:18:52.294185+0000 mgr.y (mgr.24475) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:54.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:53 smithi105 bash[19734]: audit 2024-02-18T01:18:53.792181+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:54.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:53 smithi155 bash[20805]: cluster 2024-02-18T01:18:52.294185+0000 mgr.y (mgr.24475) 73 : cluster [DBG] pgmap v43: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 0 B/s wr, 4 op/s; 9 B/s, 1 objects/s recovering 2024-02-18T01:18:54.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:53 smithi155 bash[20805]: audit 2024-02-18T01:18:53.792181+0000 mon.a (mon.0) 699 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: audit 2024-02-18T01:18:54.258274+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: cluster 2024-02-18T01:18:54.294987+0000 mgr.y (mgr.24475) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: audit 2024-02-18T01:18:54.552523+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: audit 2024-02-18T01:18:54.554570+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: audit 2024-02-18T01:18:54.565223+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[19734]: audit 2024-02-18T01:18:54.577747+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: audit 2024-02-18T01:18:54.258274+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: cluster 2024-02-18T01:18:54.294987+0000 mgr.y (mgr.24475) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: audit 2024-02-18T01:18:54.552523+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: audit 2024-02-18T01:18:54.554570+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: audit 2024-02-18T01:18:54.565223+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:55 smithi105 bash[27669]: audit 2024-02-18T01:18:54.577747+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: audit 2024-02-18T01:18:54.258274+0000 mon.a (mon.0) 700 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: cluster 2024-02-18T01:18:54.294987+0000 mgr.y (mgr.24475) 74 : cluster [DBG] pgmap v44: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: audit 2024-02-18T01:18:54.552523+0000 mon.a (mon.0) 701 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: audit 2024-02-18T01:18:54.554570+0000 mon.a (mon.0) 702 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: audit 2024-02-18T01:18:54.565223+0000 mon.a (mon.0) 703 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:18:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:55 smithi155 bash[20805]: audit 2024-02-18T01:18:54.577747+0000 mon.a (mon.0) 704 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:18:55.891 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:18:56.942 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:18:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:18:56] "GET /metrics HTTP/1.1" 200 172155 "" "Prometheus/2.33.4" 2024-02-18T01:18:57.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:57 smithi105 bash[19734]: cluster 2024-02-18T01:18:56.295687+0000 mgr.y (mgr.24475) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:57.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:57 smithi105 bash[27669]: cluster 2024-02-18T01:18:56.295687+0000 mgr.y (mgr.24475) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:57.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:18:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:18:57.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:57 smithi155 bash[20805]: cluster 2024-02-18T01:18:56.295687+0000 mgr.y (mgr.24475) 75 : cluster [DBG] pgmap v45: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:18:58.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:58 smithi105 bash[19734]: audit 2024-02-18T01:18:57.327918+0000 mgr.y (mgr.24475) 76 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:58 smithi105 bash[27669]: audit 2024-02-18T01:18:57.327918+0000 mgr.y (mgr.24475) 76 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:58 smithi155 bash[20805]: audit 2024-02-18T01:18:57.327918+0000 mgr.y (mgr.24475) 76 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:18:58.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.974+0000 7f6b7d775700 1 -- 172.21.15.105:0/758201683 >> v1:172.21.15.105:6789/0 conn(0x7f6b780fca30 legacy=0x7f6b780fce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:58.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.974+0000 7f6b7d775700 1 -- 172.21.15.105:0/758201683 shutdown_connections 2024-02-18T01:18:58.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.974+0000 7f6b7d775700 1 -- 172.21.15.105:0/758201683 >> 172.21.15.105:0/758201683 conn(0x7f6b7806fef0 msgr2=0x7f6b780f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:58.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.974+0000 7f6b7d775700 1 -- 172.21.15.105:0/758201683 shutdown_connections 2024-02-18T01:18:58.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.974+0000 7f6b7d775700 1 -- 172.21.15.105:0/758201683 wait complete. 2024-02-18T01:18:58.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 Processor -- start 2024-02-18T01:18:58.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- start start 2024-02-18T01:18:58.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6b7810ea70 con 0x7f6b78103380 2024-02-18T01:18:58.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6b7810ec00 con 0x7f6b780fca30 2024-02-18T01:18:58.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f6b7811fd80 con 0x7f6b780ff810 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b767fc700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f6b780ff810 0x7f6b780f6750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:57998/0 (socket says 172.21.15.105:57998) 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b767fc700 1 -- 172.21.15.105:0/2561363148 learned_addr learned my addr 172.21.15.105:0/2561363148 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896144844 0 0) 0x7f6b7811fd80 con 0x7f6b780ff810 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b640033f0 con 0x7f6b780ff810 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1287819432 0 0) 0x7f6b7810ea70 con 0x7f6b78103380 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7811fd80 con 0x7f6b78103380 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2233575190 0 0) 0x7f6b7810ec00 con 0x7f6b780fca30 2024-02-18T01:18:58.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b7810ea70 con 0x7f6b780fca30 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1333218766 0 0) 0x7f6b7811fd80 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b7810ec00 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6b70001c50 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1228996831 0 0) 0x7f6b640033f0 con 0x7f6b780ff810 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f6b7811fd80 con 0x7f6b780ff810 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6b60001900 con 0x7f6b780ff810 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1119680616 0 0) 0x7f6b7810ec00 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 >> v1:172.21.15.105:6790/0 conn(0x7f6b780ff810 legacy=0x7f6b780f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 >> v1:172.21.15.155:6789/0 conn(0x7f6b780fca30 legacy=0x7f6b780f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b78120fa0 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7811ff70 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f6b70002610 con 0x7f6b78103380 2024-02-18T01:18:58.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f6b70003580 con 0x7f6b78103380 2024-02-18T01:18:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.978+0000 7f6b7d775700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b781203e0 con 0x7f6b78103380 2024-02-18T01:18:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.982+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7805ecb0 con 0x7f6b78103380 2024-02-18T01:18:58.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.982+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f6b70002170 con 0x7f6b78103380 2024-02-18T01:18:58.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.982+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f6b700760a0 con 0x7f6b78103380 2024-02-18T01:18:58.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:58.986+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f6b70049c10 con 0x7f6b78103380 2024-02-18T01:18:59.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"} v 0) v1 -- 0x7f6b78109090 con 0x7f6b78103380 2024-02-18T01:18:59.214 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6ffff700 1 -- 172.21.15.105:0/2561363148 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]=0 v64) v1 ==== 105+0+11 (unknown 1088438545 0 3519293104) 0x7f6b70048020 con 0x7f6b78103380 2024-02-18T01:18:59.215 INFO:teuthology.orchestra.run.smithi105.stdout:pg_num: 32 2024-02-18T01:18:59.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 >> v1:172.21.15.105:6800/2148176012 conn(0x7f6b64062020 legacy=0x7f6b640644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:59.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 >> v1:172.21.15.105:6789/0 conn(0x7f6b78103380 legacy=0x7f6b7811c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:18:59.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 shutdown_connections 2024-02-18T01:18:59.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 >> 172.21.15.105:0/2561363148 conn(0x7f6b7806fef0 msgr2=0x7f6b780f4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:18:59.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 shutdown_connections 2024-02-18T01:18:59.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:18:59.210+0000 7f6b6dffb700 1 -- 172.21.15.105:0/2561363148 wait complete. 2024-02-18T01:18:59.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:59 smithi105 bash[19734]: cluster 2024-02-18T01:18:58.297143+0000 mgr.y (mgr.24475) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:59.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:18:59 smithi105 bash[19734]: audit 2024-02-18T01:18:59.214055+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? v1:172.21.15.105:0/2561363148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T01:18:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:59 smithi105 bash[27669]: cluster 2024-02-18T01:18:58.297143+0000 mgr.y (mgr.24475) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:18:59 smithi105 bash[27669]: audit 2024-02-18T01:18:59.214055+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? v1:172.21.15.105:0/2561363148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T01:18:59.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:59 smithi155 bash[20805]: cluster 2024-02-18T01:18:58.297143+0000 mgr.y (mgr.24475) 77 : cluster [DBG] pgmap v46: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:18:59.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:18:59 smithi155 bash[20805]: audit 2024-02-18T01:18:59.214055+0000 mon.a (mon.0) 705 : audit [DBG] from='client.? v1:172.21.15.105:0/2561363148' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "default.rgw.meta", "var": "pg_num"}]: dispatch 2024-02-18T01:18:59.973 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-02-18T01:18:59.973 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-02-18T01:18:59.973 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph mgr dump --format=json 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: audit 2024-02-18T01:18:59.997144+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: cluster 2024-02-18T01:19:00.297991+0000 mgr.y (mgr.24475) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: audit 2024-02-18T01:19:00.858523+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: audit 2024-02-18T01:19:00.859557+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: audit 2024-02-18T01:19:00.860773+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[19734]: audit 2024-02-18T01:19:00.868219+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: audit 2024-02-18T01:18:59.997144+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: cluster 2024-02-18T01:19:00.297991+0000 mgr.y (mgr.24475) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: audit 2024-02-18T01:19:00.858523+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: audit 2024-02-18T01:19:00.859557+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:19:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: audit 2024-02-18T01:19:00.860773+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:19:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:00 smithi105 bash[27669]: audit 2024-02-18T01:19:00.868219+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: audit 2024-02-18T01:18:59.997144+0000 mon.a (mon.0) 706 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: cluster 2024-02-18T01:19:00.297991+0000 mgr.y (mgr.24475) 78 : cluster [DBG] pgmap v47: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: audit 2024-02-18T01:19:00.858523+0000 mon.a (mon.0) 707 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: audit 2024-02-18T01:19:00.859557+0000 mon.a (mon.0) 708 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:19:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: audit 2024-02-18T01:19:00.860773+0000 mon.a (mon.0) 709 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:19:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:00 smithi155 bash[20805]: audit 2024-02-18T01:19:00.868219+0000 mon.a (mon.0) 710 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:19:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:03.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:03 smithi105 bash[19734]: cluster 2024-02-18T01:19:02.299440+0000 mgr.y (mgr.24475) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:03.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:03 smithi105 bash[27669]: cluster 2024-02-18T01:19:02.299440+0000 mgr.y (mgr.24475) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:03.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:03 smithi155 bash[20805]: cluster 2024-02-18T01:19:02.299440+0000 mgr.y (mgr.24475) 79 : cluster [DBG] pgmap v48: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:04.813 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:05.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:05 smithi105 bash[19734]: cluster 2024-02-18T01:19:04.299976+0000 mgr.y (mgr.24475) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:05.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:05 smithi105 bash[27669]: cluster 2024-02-18T01:19:04.299976+0000 mgr.y (mgr.24475) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:05.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:05 smithi155 bash[20805]: cluster 2024-02-18T01:19:04.299976+0000 mgr.y (mgr.24475) 80 : cluster [DBG] pgmap v49: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:06] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:19:07.738 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:07 smithi105 bash[19734]: cluster 2024-02-18T01:19:06.300681+0000 mgr.y (mgr.24475) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:07.738 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:07 smithi105 bash[27669]: cluster 2024-02-18T01:19:06.300681+0000 mgr.y (mgr.24475) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:07 smithi155 bash[20805]: cluster 2024-02-18T01:19:06.300681+0000 mgr.y (mgr.24475) 81 : cluster [DBG] pgmap v50: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:07.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:07.978 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- 172.21.15.105:0/885996497 >> v1:172.21.15.105:6790/0 conn(0x7f13dc1043a0 legacy=0x7f13dc106830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:07.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- 172.21.15.105:0/885996497 shutdown_connections 2024-02-18T01:19:07.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- 172.21.15.105:0/885996497 >> 172.21.15.105:0/885996497 conn(0x7f13dc0f3630 msgr2=0x7f13dc0f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:07.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- 172.21.15.105:0/885996497 shutdown_connections 2024-02-18T01:19:07.979 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- 172.21.15.105:0/885996497 wait complete. 2024-02-18T01:19:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 Processor -- start 2024-02-18T01:19:07.980 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- start start 2024-02-18T01:19:07.981 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f13dc117390 con 0x7f13dc100890 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f13dc118590 con 0x7f13dc1043a0 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e33a9700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f13dc119790 con 0x7f13dc0fda40 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e1145700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f13dc0fda40 0x7f13dc0f9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:37834/0 (socket says 172.21.15.105:37834) 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13e1145700 1 -- 172.21.15.105:0/2293824235 learned_addr learned my addr 172.21.15.105:0/2293824235 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291668478 0 0) 0x7f13dc119790 con 0x7f13dc0fda40 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.974+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13c8003110 con 0x7f13dc0fda40 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059570518 0 0) 0x7f13dc117390 con 0x7f13dc100890 2024-02-18T01:19:07.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13dc119790 con 0x7f13dc100890 2024-02-18T01:19:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959817930 0 0) 0x7f13dc118590 con 0x7f13dc1043a0 2024-02-18T01:19:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13dc117390 con 0x7f13dc1043a0 2024-02-18T01:19:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2481503446 0 0) 0x7f13c8003110 con 0x7f13dc0fda40 2024-02-18T01:19:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13dc118590 con 0x7f13dc0fda40 2024-02-18T01:19:07.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f13d80019f0 con 0x7f13dc0fda40 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1070487107 0 0) 0x7f13dc119790 con 0x7f13dc100890 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13c8003110 con 0x7f13dc100890 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f13cc001880 con 0x7f13dc100890 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 518813118 0 0) 0x7f13dc118590 con 0x7f13dc0fda40 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 >> v1:172.21.15.155:6789/0 conn(0x7f13dc1043a0 legacy=0x7f13dc115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 >> v1:172.21.15.105:6789/0 conn(0x7f13dc100890 legacy=0x7f13dc0fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc11a990 con 0x7f13dc0fda40 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc117580 con 0x7f13dc0fda40 2024-02-18T01:19:07.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1179f0 con 0x7f13dc0fda40 2024-02-18T01:19:07.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f13d8002410 con 0x7f13dc0fda40 2024-02-18T01:19:07.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f13d8005a70 con 0x7f13dc0fda40 2024-02-18T01:19:07.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13c0004f20 con 0x7f13dc0fda40 2024-02-18T01:19:07.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.978+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f13d80189e0 con 0x7f13dc0fda40 2024-02-18T01:19:07.987 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.982+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f13d8078600 con 0x7f13dc0fda40 2024-02-18T01:19:07.992 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:07.986+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f13d804c1d0 con 0x7f13dc0fda40 2024-02-18T01:19:08.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.218+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f13c00051d0 con 0x7f13dc0fda40 2024-02-18T01:19:08.230 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.226+0000 7f13d27fc700 1 -- 172.21.15.105:0/2293824235 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+151943 (unknown 3383617513 0 1832024755) 0x7f13d804a5e0 con 0x7f13dc0fda40 2024-02-18T01:19:08.233 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 >> v1:172.21.15.105:6800/2148176012 conn(0x7f13c80621e0 legacy=0x7f13c80646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 >> v1:172.21.15.105:6790/0 conn(0x7f13dc0fda40 legacy=0x7f13dc0f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 shutdown_connections 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 >> 172.21.15.105:0/2293824235 conn(0x7f13dc0f3630 msgr2=0x7f13dc107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 shutdown_connections 2024-02-18T01:19:08.235 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:08.230+0000 7f13e33a9700 1 -- 172.21.15.105:0/2293824235 wait complete. 2024-02-18T01:19:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:08 smithi105 bash[19734]: audit 2024-02-18T01:19:07.332958+0000 mgr.y (mgr.24475) 82 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:08 smithi105 bash[19734]: audit 2024-02-18T01:19:08.223193+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? v1:172.21.15.105:0/2293824235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T01:19:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:08 smithi105 bash[27669]: audit 2024-02-18T01:19:07.332958+0000 mgr.y (mgr.24475) 82 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:08 smithi105 bash[27669]: audit 2024-02-18T01:19:08.223193+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? v1:172.21.15.105:0/2293824235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T01:19:08.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:08 smithi155 bash[20805]: audit 2024-02-18T01:19:07.332958+0000 mgr.y (mgr.24475) 82 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:08 smithi155 bash[20805]: audit 2024-02-18T01:19:08.223193+0000 mon.c (mon.2) 27 : audit [DBG] from='client.? v1:172.21.15.105:0/2293824235' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-02-18T01:19:08.882 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":19,"active_gid":24475,"active_name":"y","active_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6800","nonce":2148176012}]},"active_addr":"172.21.15.105:6800/2148176012","active_change":"2024-02-18T01:17:36.238381+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":24469,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.105:8443/","prometheus":"http://172.21.15.105:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":62,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2457425123}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.105:0","nonce":2320425294}]}]} 2024-02-18T01:19:08.886 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-02-18T01:19:08.886 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-02-18T01:19:08.886 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T01:19:09.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:09 smithi105 bash[19734]: cluster 2024-02-18T01:19:08.302217+0000 mgr.y (mgr.24475) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:09.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:09 smithi105 bash[27669]: cluster 2024-02-18T01:19:08.302217+0000 mgr.y (mgr.24475) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:09.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:09 smithi155 bash[20805]: cluster 2024-02-18T01:19:08.302217+0000 mgr.y (mgr.24475) 83 : cluster [DBG] pgmap v51: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:11 smithi105 bash[19734]: cluster 2024-02-18T01:19:10.302927+0000 mgr.y (mgr.24475) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:11 smithi105 bash[27669]: cluster 2024-02-18T01:19:10.302927+0000 mgr.y (mgr.24475) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:11.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:11 smithi155 bash[20805]: cluster 2024-02-18T01:19:10.302927+0000 mgr.y (mgr.24475) 84 : cluster [DBG] pgmap v52: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:13.726 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:13.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:13 smithi105 bash[19734]: cluster 2024-02-18T01:19:12.304273+0000 mgr.y (mgr.24475) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:13 smithi105 bash[27669]: cluster 2024-02-18T01:19:12.304273+0000 mgr.y (mgr.24475) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:13 smithi155 bash[20805]: cluster 2024-02-18T01:19:12.304273+0000 mgr.y (mgr.24475) 85 : cluster [DBG] pgmap v53: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:15.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:15 smithi105 bash[19734]: cluster 2024-02-18T01:19:14.304987+0000 mgr.y (mgr.24475) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:15.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:15 smithi105 bash[27669]: cluster 2024-02-18T01:19:14.304987+0000 mgr.y (mgr.24475) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:15.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:15 smithi155 bash[20805]: cluster 2024-02-18T01:19:14.304987+0000 mgr.y (mgr.24475) 86 : cluster [DBG] pgmap v54: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- 172.21.15.105:0/504162821 >> v1:172.21.15.105:6790/0 conn(0x7efe2c100890 legacy=0x7efe2c102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:16.760 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- 172.21.15.105:0/504162821 shutdown_connections 2024-02-18T01:19:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- 172.21.15.105:0/504162821 >> 172.21.15.105:0/504162821 conn(0x7efe2c06c140 msgr2=0x7efe2c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- 172.21.15.105:0/504162821 shutdown_connections 2024-02-18T01:19:16.761 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- 172.21.15.105:0/504162821 wait complete. 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 Processor -- start 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.753+0000 7efe327d0700 1 -- start start 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe2c1173b0 con 0x7efe2c100890 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe2c118590 con 0x7efe2c1043d0 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efe2c119790 con 0x7efe2c0fda60 2024-02-18T01:19:16.762 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe2bfff700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7efe2c0fda60 0x7efe2c0f9b20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:60584/0 (socket says 172.21.15.105:60584) 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe2bfff700 1 -- 172.21.15.105:0/3197180787 learned_addr learned my addr 172.21.15.105:0/3197180787 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe2b7fe700 1 --1- 172.21.15.105:0/3197180787 >> v1:172.21.15.105:6789/0 conn(0x7efe2c100890 0x7efe2c0fa230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54948/0 (socket says 172.21.15.105:54948) 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024377680 0 0) 0x7efe2c1173b0 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe140033f0 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1411631091 0 0) 0x7efe2c119790 con 0x7efe2c0fda60 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe2c1173b0 con 0x7efe2c0fda60 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1702212123 0 0) 0x7efe140033f0 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efe2c119790 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7efe20001430 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1006764503 0 0) 0x7efe2c118590 con 0x7efe2c1043d0 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe140033f0 con 0x7efe2c1043d0 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 598349945 0 0) 0x7efe2c119790 con 0x7efe2c100890 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 >> v1:172.21.15.105:6790/0 conn(0x7efe2c0fda60 legacy=0x7efe2c0f9b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:16.763 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 >> v1:172.21.15.155:6789/0 conn(0x7efe2c1043d0 legacy=0x7efe2c115a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c11a990 con 0x7efe2c100890 2024-02-18T01:19:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c119980 con 0x7efe2c100890 2024-02-18T01:19:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c119df0 con 0x7efe2c100890 2024-02-18T01:19:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7efe20001f70 con 0x7efe2c100890 2024-02-18T01:19:16.764 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7efe20003800 con 0x7efe2c100890 2024-02-18T01:19:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7efe20003960 con 0x7efe2c100890 2024-02-18T01:19:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.757+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe18004f50 con 0x7efe2c100890 2024-02-18T01:19:16.765 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.761+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7efe200788f0 con 0x7efe2c100890 2024-02-18T01:19:16.771 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.765+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7efe2004c580 con 0x7efe2c100890 2024-02-18T01:19:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.953+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe18005200 con 0x7efe2c100890 2024-02-18T01:19:16.961 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.957+0000 7efe297fa700 1 -- 172.21.15.105:0/3197180787 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19204 (unknown 2319294364 0 2768765582) 0x7efe2004a9f0 con 0x7efe2c100890 2024-02-18T01:19:16.962 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:19:16.962 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":64,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","created":"2024-02-18T01:08:57.371365+0000","modified":"2024-02-18T01:18:37.541981+0000","last_up_change":"2024-02-18T01:16:55.358934+0000","last_in_change":"2024-02-18T01:16:29.814709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T01:13:29.045566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-18T01:17:08.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-18T01:17:09.343824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T01:17:11.795816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T01:17:13.251513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T01:17:15.371360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"62250a45-b3fb-4078-8edb-dc54138a7b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6801","nonce":3115838352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6802","nonce":3115838352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6804","nonce":3115838352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6803","nonce":3115838352}]},"public_addr":"172.21.15.105:6801/3115838352","cluster_addr":"172.21.15.105:6802/3115838352","heartbeat_back_addr":"172.21.15.105:6804/3115838352","heartbeat_front_addr":"172.21.15.105:6803/3115838352","state":["exists","up"]},{"osd":1,"uuid":"59487fa0-277f-4175-b49b-f50ee2645931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6805","nonce":1532722979}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6806","nonce":1532722979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6808","nonce":1532722979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6807","nonce":1532722979}]},"public_addr":"172.21.15.105:6805/1532722979","cluster_addr":"172.21.15.105:6806/1532722979","heartbeat_back_addr":"172.21.15.105:6808/1532722979","heartbeat_front_addr":"172.21.15.105:6807/1532722979","state":["exists","up"]},{"osd":2,"uuid":"c08740fc-b607-4226-be20-1c56854e971c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6809","nonce":3935238244}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6810","nonce":3935238244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6812","nonce":3935238244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6811","nonce":3935238244}]},"public_addr":"172.21.15.105:6809/3935238244","cluster_addr":"172.21.15.105:6810/3935238244","heartbeat_back_addr":"172.21.15.105:6812/3935238244","heartbeat_front_addr":"172.21.15.105:6811/3935238244","state":["exists","up"]},{"osd":3,"uuid":"eaef36ae-a5ea-4a4a-8c5f-c88098a48611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6813","nonce":1099653553}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6814","nonce":1099653553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6816","nonce":1099653553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6815","nonce":1099653553}]},"public_addr":"172.21.15.105:6813/1099653553","cluster_addr":"172.21.15.105:6814/1099653553","heartbeat_back_addr":"172.21.15.105:6816/1099653553","heartbeat_front_addr":"172.21.15.105:6815/1099653553","state":["exists","up"]},{"osd":4,"uuid":"21226459-02ce-41a2-8761-c8125feea431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1608438726}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1608438726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1608438726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1608438726}]},"public_addr":"172.21.15.155:6800/1608438726","cluster_addr":"172.21.15.155:6801/1608438726","heartbeat_back_addr":"172.21.15.155:6803/1608438726","heartbeat_front_addr":"172.21.15.155:6802/1608438726","state":["exists","up"]},{"osd":5,"uuid":"2650bfc0-0a69-4425-aaef-31bada765f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":3674587623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":3674587623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":3674587623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":3674587623}]},"public_addr":"172.21.15.155:6804/3674587623","cluster_addr":"172.21.15.155:6805/3674587623","heartbeat_back_addr":"172.21.15.155:6807/3674587623","heartbeat_front_addr":"172.21.15.155:6806/3674587623","state":["exists","up"]},{"osd":6,"uuid":"565ca7a5-3796-4f8a-93be-8e98f818ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":98819544}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":98819544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":98819544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":98819544}]},"public_addr":"172.21.15.155:6808/98819544","cluster_addr":"172.21.15.155:6809/98819544","heartbeat_back_addr":"172.21.15.155:6811/98819544","heartbeat_front_addr":"172.21.15.155:6810/98819544","state":["exists","up"]},{"osd":7,"uuid":"0b9beb5b-519d-4ef0-9768-cf509db5d07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":1643073216}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":1643073216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":1643073216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":1643073216}]},"public_addr":"172.21.15.155:6812/1643073216","cluster_addr":"172.21.15.155:6813/1643073216","heartbeat_back_addr":"172.21.15.155:6815/1643073216","heartbeat_front_addr":"172.21.15.155:6814/1643073216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:03.400874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:44.855453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:13:26.607272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:09.302207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:48.550664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:15:30.305002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:11.430016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:53.541617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6800/3087430119":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/4165234016":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/3637238460":"2024-02-19T01:09:55.434898+0000","172.21.15.105:0/2779353230":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/1813807489":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/3260915113":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/2780429744":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/1333885900":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/2401194680":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/999225556":"2024-02-19T01:09:25.603699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T01:19:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.957+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 >> v1:172.21.15.105:6800/2148176012 conn(0x7efe14061fb0 legacy=0x7efe14064470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.957+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 >> v1:172.21.15.105:6789/0 conn(0x7efe2c100890 legacy=0x7efe2c0fa230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.961+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 shutdown_connections 2024-02-18T01:19:16.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.961+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 >> 172.21.15.105:0/3197180787 conn(0x7efe2c06c140 msgr2=0x7efe2c06d500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:16.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.961+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 shutdown_connections 2024-02-18T01:19:16.966 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:16.961+0000 7efe327d0700 1 -- 172.21.15.105:0/3197180787 wait complete. 2024-02-18T01:19:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:16] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:19:17.635 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-02-18T01:19:17.635 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd dump --format=json 2024-02-18T01:19:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:17 smithi105 bash[19734]: cluster 2024-02-18T01:19:16.305673+0000 mgr.y (mgr.24475) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:17.646 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:17 smithi105 bash[19734]: audit 2024-02-18T01:19:16.959616+0000 mon.a (mon.0) 711 : audit [DBG] from='client.? v1:172.21.15.105:0/3197180787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:17.647 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:17 smithi105 bash[27669]: cluster 2024-02-18T01:19:16.305673+0000 mgr.y (mgr.24475) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:17.647 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:17 smithi105 bash[27669]: audit 2024-02-18T01:19:16.959616+0000 mon.a (mon.0) 711 : audit [DBG] from='client.? v1:172.21.15.105:0/3197180787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:17.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:17.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:17 smithi155 bash[20805]: cluster 2024-02-18T01:19:16.305673+0000 mgr.y (mgr.24475) 87 : cluster [DBG] pgmap v55: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:17.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:17 smithi155 bash[20805]: audit 2024-02-18T01:19:16.959616+0000 mon.a (mon.0) 711 : audit [DBG] from='client.? v1:172.21.15.105:0/3197180787' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:18.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:18 smithi105 bash[19734]: audit 2024-02-18T01:19:17.343833+0000 mgr.y (mgr.24475) 88 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:18 smithi105 bash[27669]: audit 2024-02-18T01:19:17.343833+0000 mgr.y (mgr.24475) 88 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:18 smithi155 bash[20805]: audit 2024-02-18T01:19:17.343833+0000 mgr.y (mgr.24475) 88 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:19.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:19 smithi105 bash[19734]: cluster 2024-02-18T01:19:18.307072+0000 mgr.y (mgr.24475) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:19.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:19 smithi105 bash[27669]: cluster 2024-02-18T01:19:18.307072+0000 mgr.y (mgr.24475) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:19 smithi155 bash[20805]: cluster 2024-02-18T01:19:18.307072+0000 mgr.y (mgr.24475) 89 : cluster [DBG] pgmap v56: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:21 smithi105 bash[19734]: cluster 2024-02-18T01:19:20.307778+0000 mgr.y (mgr.24475) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:21 smithi105 bash[27669]: cluster 2024-02-18T01:19:20.307778+0000 mgr.y (mgr.24475) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:21 smithi155 bash[20805]: cluster 2024-02-18T01:19:20.307778+0000 mgr.y (mgr.24475) 90 : cluster [DBG] pgmap v57: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:22.472 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:23.649 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:23 smithi105 bash[27669]: cluster 2024-02-18T01:19:22.309133+0000 mgr.y (mgr.24475) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:23.649 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:23 smithi105 bash[19734]: cluster 2024-02-18T01:19:22.309133+0000 mgr.y (mgr.24475) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:23 smithi155 bash[20805]: cluster 2024-02-18T01:19:22.309133+0000 mgr.y (mgr.24475) 91 : cluster [DBG] pgmap v58: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:25.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- 172.21.15.105:0/3872696555 >> v1:172.21.15.105:6790/0 conn(0x7f13200fda40 legacy=0x7f13200fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:25.334 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- 172.21.15.105:0/3872696555 shutdown_connections 2024-02-18T01:19:25.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- 172.21.15.105:0/3872696555 >> 172.21.15.105:0/3872696555 conn(0x7f13200f3630 msgr2=0x7f13200f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:25.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- 172.21.15.105:0/3872696555 shutdown_connections 2024-02-18T01:19:25.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- 172.21.15.105:0/3872696555 wait complete. 2024-02-18T01:19:25.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 Processor -- start 2024-02-18T01:19:25.335 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- start start 2024-02-18T01:19:25.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1320117390 con 0x7f13201043a0 2024-02-18T01:19:25.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1320118590 con 0x7f1320100890 2024-02-18T01:19:25.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1327276700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1320119790 con 0x7f13200fda40 2024-02-18T01:19:25.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1325012700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f13200fda40 0x7f13200f9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:41626/0 (socket says 172.21.15.105:41626) 2024-02-18T01:19:25.336 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1325012700 1 -- 172.21.15.105:0/2441917304 learned_addr learned my addr 172.21.15.105:0/2441917304 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:25.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2628473608 0 0) 0x7f1320119790 con 0x7f13200fda40 2024-02-18T01:19:25.337 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f130c0033f0 con 0x7f13200fda40 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 247226425 0 0) 0x7f1320117390 con 0x7f13201043a0 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320119790 con 0x7f13201043a0 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3240253259 0 0) 0x7f1320118590 con 0x7f1320100890 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1320117390 con 0x7f1320100890 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3945137028 0 0) 0x7f130c0033f0 con 0x7f13200fda40 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1320118590 con 0x7f13200fda40 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f131c0023d0 con 0x7f13200fda40 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3698565779 0 0) 0x7f1320119790 con 0x7f13201043a0 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f130c0033f0 con 0x7f13201043a0 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.329+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f13180017c0 con 0x7f13201043a0 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2811299948 0 0) 0x7f1320118590 con 0x7f13200fda40 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 >> v1:172.21.15.155:6789/0 conn(0x7f1320100890 legacy=0x7f13200fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 >> v1:172.21.15.105:6789/0 conn(0x7f13201043a0 legacy=0x7f1320115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:25.338 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f132011a990 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1320117580 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f13201179f0 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f131c0031e0 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f131c003bb0 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1304004f20 con 0x7f13200fda40 2024-02-18T01:19:25.339 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.333+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f131c003d10 con 0x7f13200fda40 2024-02-18T01:19:25.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.337+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f131c078ec0 con 0x7f13200fda40 2024-02-18T01:19:25.351 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.345+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f131c04d6d0 con 0x7f13200fda40 2024-02-18T01:19:25.539 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.533+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13040051d0 con 0x7f13200fda40 2024-02-18T01:19:25.541 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.533+0000 7f1312777700 1 -- 172.21.15.105:0/2441917304 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+19204 (unknown 2319294364 0 2768765582) 0x7f131c04bb40 con 0x7f13200fda40 2024-02-18T01:19:25.542 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:19:25.542 INFO:teuthology.orchestra.run.smithi105.stdout:{"epoch":64,"fsid":"358e970a-cdfa-11ee-95ba-87774f69a715","created":"2024-02-18T01:08:57.371365+0000","modified":"2024-02-18T01:18:37.541981+0000","last_up_change":"2024-02-18T01:16:55.358934+0000","last_in_change":"2024-02-18T01:16:29.814709+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2024-02-18T01:13:29.045566+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"iscsi","create_time":"2024-02-18T01:17:08.301344+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":".rgw.root","create_time":"2024-02-18T01:17:09.343824+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-02-18T01:17:11.795816+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-02-18T01:17:13.251513+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-02-18T01:17:15.371360+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"62250a45-b3fb-4078-8edb-dc54138a7b9f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6801","nonce":3115838352}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6802","nonce":3115838352}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6804","nonce":3115838352}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6803","nonce":3115838352}]},"public_addr":"172.21.15.105:6801/3115838352","cluster_addr":"172.21.15.105:6802/3115838352","heartbeat_back_addr":"172.21.15.105:6804/3115838352","heartbeat_front_addr":"172.21.15.105:6803/3115838352","state":["exists","up"]},{"osd":1,"uuid":"59487fa0-277f-4175-b49b-f50ee2645931","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6805","nonce":1532722979}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6806","nonce":1532722979}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6808","nonce":1532722979}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6807","nonce":1532722979}]},"public_addr":"172.21.15.105:6805/1532722979","cluster_addr":"172.21.15.105:6806/1532722979","heartbeat_back_addr":"172.21.15.105:6808/1532722979","heartbeat_front_addr":"172.21.15.105:6807/1532722979","state":["exists","up"]},{"osd":2,"uuid":"c08740fc-b607-4226-be20-1c56854e971c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6809","nonce":3935238244}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6810","nonce":3935238244}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6812","nonce":3935238244}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6811","nonce":3935238244}]},"public_addr":"172.21.15.105:6809/3935238244","cluster_addr":"172.21.15.105:6810/3935238244","heartbeat_back_addr":"172.21.15.105:6812/3935238244","heartbeat_front_addr":"172.21.15.105:6811/3935238244","state":["exists","up"]},{"osd":3,"uuid":"eaef36ae-a5ea-4a4a-8c5f-c88098a48611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6813","nonce":1099653553}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6814","nonce":1099653553}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6816","nonce":1099653553}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.105:6815","nonce":1099653553}]},"public_addr":"172.21.15.105:6813/1099653553","cluster_addr":"172.21.15.105:6814/1099653553","heartbeat_back_addr":"172.21.15.105:6816/1099653553","heartbeat_front_addr":"172.21.15.105:6815/1099653553","state":["exists","up"]},{"osd":4,"uuid":"21226459-02ce-41a2-8761-c8125feea431","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6800","nonce":1608438726}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6801","nonce":1608438726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6803","nonce":1608438726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6802","nonce":1608438726}]},"public_addr":"172.21.15.155:6800/1608438726","cluster_addr":"172.21.15.155:6801/1608438726","heartbeat_back_addr":"172.21.15.155:6803/1608438726","heartbeat_front_addr":"172.21.15.155:6802/1608438726","state":["exists","up"]},{"osd":5,"uuid":"2650bfc0-0a69-4425-aaef-31bada765f29","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6804","nonce":3674587623}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6805","nonce":3674587623}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6807","nonce":3674587623}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6806","nonce":3674587623}]},"public_addr":"172.21.15.155:6804/3674587623","cluster_addr":"172.21.15.155:6805/3674587623","heartbeat_back_addr":"172.21.15.155:6807/3674587623","heartbeat_front_addr":"172.21.15.155:6806/3674587623","state":["exists","up"]},{"osd":6,"uuid":"565ca7a5-3796-4f8a-93be-8e98f818ff85","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6808","nonce":98819544}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6809","nonce":98819544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6811","nonce":98819544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6810","nonce":98819544}]},"public_addr":"172.21.15.155:6808/98819544","cluster_addr":"172.21.15.155:6809/98819544","heartbeat_back_addr":"172.21.15.155:6811/98819544","heartbeat_front_addr":"172.21.15.155:6810/98819544","state":["exists","up"]},{"osd":7,"uuid":"0b9beb5b-519d-4ef0-9768-cf509db5d07d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6812","nonce":1643073216}]},"cluster_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6813","nonce":1643073216}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6815","nonce":1643073216}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v1","addr":"172.21.15.155:6814","nonce":1643073216}]},"public_addr":"172.21.15.155:6812/1643073216","cluster_addr":"172.21.15.155:6813/1643073216","heartbeat_back_addr":"172.21.15.155:6815/1643073216","heartbeat_front_addr":"172.21.15.155:6814/1643073216","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:03.400874+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:12:44.855453+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:13:26.607272+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:09.302207+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:14:48.550664+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:15:30.305002+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:11.430016+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2024-02-18T01:16:53.541617+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.6","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.105:6800/3087430119":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/4165234016":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/3637238460":"2024-02-19T01:09:55.434898+0000","172.21.15.105:0/2779353230":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/1813807489":"2024-02-19T01:09:55.434898+0000","172.21.15.105:6800/3260915113":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/2780429744":"2024-02-19T01:09:25.603699+0000","172.21.15.105:0/1333885900":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/2401194680":"2024-02-19T01:17:36.238065+0000","172.21.15.105:0/999225556":"2024-02-19T01:09:25.603699+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-02-18T01:19:25.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 >> v1:172.21.15.105:6800/2148176012 conn(0x7f130c062070 legacy=0x7f130c064530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:25.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 >> v1:172.21.15.105:6790/0 conn(0x7f13200fda40 legacy=0x7f13200f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:25.543 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 shutdown_connections 2024-02-18T01:19:25.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 >> 172.21.15.105:0/2441917304 conn(0x7f13200f3630 msgr2=0x7f1320107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:25.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 shutdown_connections 2024-02-18T01:19:25.544 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:25.537+0000 7f1327276700 1 -- 172.21.15.105:0/2441917304 wait complete. 2024-02-18T01:19:25.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:25 smithi105 bash[27669]: cluster 2024-02-18T01:19:24.309822+0000 mgr.y (mgr.24475) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:25 smithi105 bash[19734]: cluster 2024-02-18T01:19:24.309822+0000 mgr.y (mgr.24475) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:25 smithi155 bash[20805]: cluster 2024-02-18T01:19:24.309822+0000 mgr.y (mgr.24475) 92 : cluster [DBG] pgmap v59: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:26.185 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-02-18T01:19:26.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:26 smithi105 bash[19734]: audit 2024-02-18T01:19:25.539099+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? v1:172.21.15.105:0/2441917304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:26.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:26 smithi105 bash[27669]: audit 2024-02-18T01:19:25.539099+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? v1:172.21.15.105:0/2441917304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:26 smithi155 bash[20805]: audit 2024-02-18T01:19:25.539099+0000 mon.c (mon.2) 28 : audit [DBG] from='client.? v1:172.21.15.105:0/2441917304' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-02-18T01:19:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:26] "GET /metrics HTTP/1.1" 200 172154 "" "Prometheus/2.33.4" 2024-02-18T01:19:27.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:27 smithi105 bash[19734]: cluster 2024-02-18T01:19:26.310514+0000 mgr.y (mgr.24475) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:27 smithi105 bash[27669]: cluster 2024-02-18T01:19:26.310514+0000 mgr.y (mgr.24475) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:27.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:27.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:27 smithi155 bash[20805]: cluster 2024-02-18T01:19:26.310514+0000 mgr.y (mgr.24475) 93 : cluster [DBG] pgmap v60: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:28 smithi105 bash[19734]: audit 2024-02-18T01:19:27.355178+0000 mgr.y (mgr.24475) 94 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:28 smithi105 bash[27669]: audit 2024-02-18T01:19:27.355178+0000 mgr.y (mgr.24475) 94 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:28 smithi155 bash[20805]: audit 2024-02-18T01:19:27.355178+0000 mgr.y (mgr.24475) 94 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:29.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:29 smithi105 bash[19734]: cluster 2024-02-18T01:19:28.311924+0000 mgr.y (mgr.24475) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:29 smithi105 bash[27669]: cluster 2024-02-18T01:19:28.311924+0000 mgr.y (mgr.24475) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:29 smithi155 bash[20805]: cluster 2024-02-18T01:19:28.311924+0000 mgr.y (mgr.24475) 95 : cluster [DBG] pgmap v61: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:31.056 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:31 smithi105 bash[19734]: cluster 2024-02-18T01:19:30.312630+0000 mgr.y (mgr.24475) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:31.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:31 smithi105 bash[27669]: cluster 2024-02-18T01:19:30.312630+0000 mgr.y (mgr.24475) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:31.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:31 smithi155 bash[20805]: cluster 2024-02-18T01:19:30.312630+0000 mgr.y (mgr.24475) 96 : cluster [DBG] pgmap v62: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:33.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:33 smithi105 bash[19734]: cluster 2024-02-18T01:19:32.314067+0000 mgr.y (mgr.24475) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:33 smithi105 bash[27669]: cluster 2024-02-18T01:19:32.314067+0000 mgr.y (mgr.24475) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:33 smithi155 bash[20805]: cluster 2024-02-18T01:19:32.314067+0000 mgr.y (mgr.24475) 97 : cluster [DBG] pgmap v63: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:34.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- 172.21.15.105:0/100473485 >> v1:172.21.15.105:6789/0 conn(0x7fed7c0fef00 legacy=0x7fed7c1012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- 172.21.15.105:0/100473485 shutdown_connections 2024-02-18T01:19:34.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- 172.21.15.105:0/100473485 >> 172.21.15.105:0/100473485 conn(0x7fed7c06c140 msgr2=0x7fed7c06c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:34.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- 172.21.15.105:0/100473485 shutdown_connections 2024-02-18T01:19:34.141 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- 172.21.15.105:0/100473485 wait complete. 2024-02-18T01:19:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 Processor -- start 2024-02-18T01:19:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.133+0000 7fed817e5700 1 -- start start 2024-02-18T01:19:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed7c119710 con 0x7fed7c0fef00 2024-02-18T01:19:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed7c11a910 con 0x7fed7c102a40 2024-02-18T01:19:34.142 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fed7c11bb10 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed73fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fed7c0fef00 0x7fed7c1144c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:49146/0 (socket says 172.21.15.105:49146) 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed73fff700 1 -- 172.21.15.105:0/1538125687 learned_addr learned my addr 172.21.15.105:0/1538125687 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251588959 0 0) 0x7fed7c119710 con 0x7fed7c0fef00 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed5c004740 con 0x7fed7c0fef00 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518796149 0 0) 0x7fed7c11bb10 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c119710 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378507066 0 0) 0x7fed7c11a910 con 0x7fed7c102a40 2024-02-18T01:19:34.143 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed7c11bb10 con 0x7fed7c102a40 2024-02-18T01:19:34.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1051513992 0 0) 0x7fed5c004740 con 0x7fed7c0fef00 2024-02-18T01:19:34.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed7c11a910 con 0x7fed7c0fef00 2024-02-18T01:19:34.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fed64002e00 con 0x7fed7c0fef00 2024-02-18T01:19:34.144 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1525193717 0 0) 0x7fed7c11bb10 con 0x7fed7c102a40 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed5c004740 con 0x7fed7c102a40 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fed74001c40 con 0x7fed7c102a40 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3189266738 0 0) 0x7fed7c119710 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fed7c11bb10 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fed6c001840 con 0x7fed7c0fc0d0 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2234090947 0 0) 0x7fed7c11a910 con 0x7fed7c0fef00 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 >> v1:172.21.15.105:6790/0 conn(0x7fed7c0fc0d0 legacy=0x7fed7c0fb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 >> v1:172.21.15.155:6789/0 conn(0x7fed7c102a40 legacy=0x7fed7c117dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c11cd10 con 0x7fed7c0fef00 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed7c11bd00 con 0x7fed7c0fef00 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fed7c11c170 con 0x7fed7c0fef00 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fed640024a0 con 0x7fed7c0fef00 2024-02-18T01:19:34.145 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fed640037c0 con 0x7fed7c0fef00 2024-02-18T01:19:34.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.137+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fed60000f10 con 0x7fed7c0fef00 2024-02-18T01:19:34.146 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.141+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fed64003a00 con 0x7fed7c0fef00 2024-02-18T01:19:34.147 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.141+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7fed64078990 con 0x7fed7c0fef00 2024-02-18T01:19:34.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.141+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6801/3115838352 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fed5c070820 con 0x7fed5c06cb10 2024-02-18T01:19:34.148 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.141+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7fed64078dc0 con 0x7fed7c0fef00 2024-02-18T01:19:34.149 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.141+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== osd.0 v1:172.21.15.105:6801/3115838352 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fed5c070820 con 0x7fed5c06cb10 2024-02-18T01:19:34.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 --> v1:172.21.15.105:6801/3115838352 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fed600029f0 con 0x7fed5c06cb10 2024-02-18T01:19:34.168 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed78ff9700 1 -- 172.21.15.105:0/1538125687 <== osd.0 v1:172.21.15.105:6801/3115838352 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2270720027) 0x7fed600029f0 con 0x7fed5c06cb10 2024-02-18T01:19:34.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 >> v1:172.21.15.105:6801/3115838352 conn(0x7fed5c06cb10 legacy=0x7fed5c06efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 >> v1:172.21.15.105:6800/2148176012 conn(0x7fed5c062340 legacy=0x7fed5c064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 >> v1:172.21.15.105:6789/0 conn(0x7fed7c0fef00 legacy=0x7fed7c1144c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:34.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed7b7fe700 1 -- 172.21.15.105:0/1538125687 reap_dead start 2024-02-18T01:19:34.169 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 shutdown_connections 2024-02-18T01:19:34.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.161+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 >> 172.21.15.105:0/1538125687 conn(0x7fed7c06c140 msgr2=0x7fed7c06c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:34.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.165+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 shutdown_connections 2024-02-18T01:19:34.170 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:34.165+0000 7fed817e5700 1 -- 172.21.15.105:0/1538125687 wait complete. 2024-02-18T01:19:34.883 INFO:teuthology.orchestra.run.smithi105.stdout:34359738459 2024-02-18T01:19:34.883 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-02-18T01:19:35.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:35 smithi105 bash[19734]: cluster 2024-02-18T01:19:34.314858+0000 mgr.y (mgr.24475) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:35.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:35 smithi105 bash[27669]: cluster 2024-02-18T01:19:34.314858+0000 mgr.y (mgr.24475) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:35.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:35 smithi155 bash[20805]: cluster 2024-02-18T01:19:34.314858+0000 mgr.y (mgr.24475) 98 : cluster [DBG] pgmap v64: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:36] "GET /metrics HTTP/1.1" 200 172153 "" "Prometheus/2.33.4" 2024-02-18T01:19:37.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:37 smithi105 bash[19734]: cluster 2024-02-18T01:19:36.315566+0000 mgr.y (mgr.24475) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:37.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:37 smithi105 bash[27669]: cluster 2024-02-18T01:19:36.315566+0000 mgr.y (mgr.24475) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:37.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:37 smithi155 bash[20805]: cluster 2024-02-18T01:19:36.315566+0000 mgr.y (mgr.24475) 99 : cluster [DBG] pgmap v65: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:38.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:38 smithi105 bash[27669]: audit 2024-02-18T01:19:37.363087+0000 mgr.y (mgr.24475) 100 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:38 smithi105 bash[19734]: audit 2024-02-18T01:19:37.363087+0000 mgr.y (mgr.24475) 100 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:38.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:38 smithi155 bash[20805]: audit 2024-02-18T01:19:37.363087+0000 mgr.y (mgr.24475) 100 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:39.714 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:39.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:39 smithi105 bash[27669]: cluster 2024-02-18T01:19:38.317040+0000 mgr.y (mgr.24475) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:39 smithi105 bash[19734]: cluster 2024-02-18T01:19:38.317040+0000 mgr.y (mgr.24475) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:39.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:39 smithi155 bash[20805]: cluster 2024-02-18T01:19:38.317040+0000 mgr.y (mgr.24475) 101 : cluster [DBG] pgmap v66: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:41.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:41 smithi105 bash[19734]: cluster 2024-02-18T01:19:40.317805+0000 mgr.y (mgr.24475) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:41.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:41 smithi105 bash[27669]: cluster 2024-02-18T01:19:40.317805+0000 mgr.y (mgr.24475) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:41.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:41 smithi155 bash[20805]: cluster 2024-02-18T01:19:40.317805+0000 mgr.y (mgr.24475) 102 : cluster [DBG] pgmap v67: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/1791499936 >> v1:172.21.15.105:6789/0 conn(0x7f807c0fda40 legacy=0x7f807c0fde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/1791499936 shutdown_connections 2024-02-18T01:19:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/1791499936 >> 172.21.15.105:0/1791499936 conn(0x7f807c0f3630 msgr2=0x7f807c0f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:42.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/1791499936 shutdown_connections 2024-02-18T01:19:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/1791499936 wait complete. 2024-02-18T01:19:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 Processor -- start 2024-02-18T01:19:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- start start 2024-02-18T01:19:42.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f807c117290 con 0x7f807c1043a0 2024-02-18T01:19:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f807c118470 con 0x7f807c100890 2024-02-18T01:19:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f807c119650 con 0x7f807c0fda40 2024-02-18T01:19:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f807affd700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f807c0fda40 0x7f807c0f98e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:42606/0 (socket says 172.21.15.105:42606) 2024-02-18T01:19:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f807affd700 1 -- 172.21.15.105:0/2237684120 learned_addr learned my addr 172.21.15.105:0/2237684120 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:42.728 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f807b7fe700 1 --1- 172.21.15.105:0/2237684120 >> v1:172.21.15.105:6789/0 conn(0x7f807c1043a0 0x7f807c115a10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:42446/0 (socket says 172.21.15.105:42446) 2024-02-18T01:19:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 152794115 0 0) 0x7f807c119650 con 0x7f807c0fda40 2024-02-18T01:19:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80680033f0 con 0x7f807c0fda40 2024-02-18T01:19:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203530620 0 0) 0x7f807c117290 con 0x7f807c1043a0 2024-02-18T01:19:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f807c119650 con 0x7f807c1043a0 2024-02-18T01:19:42.729 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4272704929 0 0) 0x7f807c118470 con 0x7f807c100890 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f807c117290 con 0x7f807c100890 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2817762327 0 0) 0x7f80680033f0 con 0x7f807c0fda40 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f807c118470 con 0x7f807c0fda40 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f806c002c70 con 0x7f807c0fda40 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2868108116 0 0) 0x7f807c117290 con 0x7f807c100890 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f80680033f0 con 0x7f807c100890 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8064001840 con 0x7f807c100890 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2718674968 0 0) 0x7f807c119650 con 0x7f807c1043a0 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f807c117290 con 0x7f807c1043a0 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8074001e80 con 0x7f807c1043a0 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1567332951 0 0) 0x7f807c118470 con 0x7f807c0fda40 2024-02-18T01:19:42.730 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 >> v1:172.21.15.155:6789/0 conn(0x7f807c100890 legacy=0x7f807c0f9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 >> v1:172.21.15.105:6789/0 conn(0x7f807c1043a0 legacy=0x7f807c115a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c11a850 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c118660 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c118ad0 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.721+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f806c001b30 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f806c003760 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f805c000f10 con 0x7f807c0fda40 2024-02-18T01:19:42.731 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f806c002dd0 con 0x7f807c0fda40 2024-02-18T01:19:42.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f806c0787a0 con 0x7f807c0fda40 2024-02-18T01:19:42.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6805/1532722979 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8068070890 con 0x7f806806cb80 2024-02-18T01:19:42.732 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.725+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7f806c078ba0 con 0x7f807c0fda40 2024-02-18T01:19:42.733 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.729+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== osd.1 v1:172.21.15.105:6805/1532722979 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f8068070890 con 0x7f806806cb80 2024-02-18T01:19:42.752 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.745+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 --> v1:172.21.15.105:6805/1532722979 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f805c0029f0 con 0x7f806806cb80 2024-02-18T01:19:42.753 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.745+0000 7f8073fff700 1 -- 172.21.15.105:0/2237684120 <== osd.1 v1:172.21.15.105:6805/1532722979 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 388304689) 0x7f805c0029f0 con 0x7f806806cb80 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 >> v1:172.21.15.105:6805/1532722979 conn(0x7f806806cb80 legacy=0x7f806806f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 >> v1:172.21.15.105:6800/2148176012 conn(0x7f8068062350 legacy=0x7f8068064810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 >> v1:172.21.15.105:6790/0 conn(0x7f807c0fda40 legacy=0x7f807c0f98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f807b7fe700 1 -- 172.21.15.105:0/2237684120 reap_dead start 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 shutdown_connections 2024-02-18T01:19:42.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 >> 172.21.15.105:0/2237684120 conn(0x7f807c0f3630 msgr2=0x7f807c107820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:42.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 shutdown_connections 2024-02-18T01:19:42.755 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:42.749+0000 7f808133b700 1 -- 172.21.15.105:0/2237684120 wait complete. 2024-02-18T01:19:43.413 INFO:teuthology.orchestra.run.smithi105.stdout:55834574933 2024-02-18T01:19:43.414 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-02-18T01:19:43.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:43 smithi105 bash[19734]: cluster 2024-02-18T01:19:42.319224+0000 mgr.y (mgr.24475) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:43 smithi105 bash[27669]: cluster 2024-02-18T01:19:42.319224+0000 mgr.y (mgr.24475) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:43.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:43 smithi155 bash[20805]: cluster 2024-02-18T01:19:42.319224+0000 mgr.y (mgr.24475) 103 : cluster [DBG] pgmap v68: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:45 smithi155 bash[20805]: cluster 2024-02-18T01:19:44.320092+0000 mgr.y (mgr.24475) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:45 smithi105 bash[19734]: cluster 2024-02-18T01:19:44.320092+0000 mgr.y (mgr.24475) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:45 smithi105 bash[27669]: cluster 2024-02-18T01:19:44.320092+0000 mgr.y (mgr.24475) 104 : cluster [DBG] pgmap v69: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:46] "GET /metrics HTTP/1.1" 200 172153 "" "Prometheus/2.33.4" 2024-02-18T01:19:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:47 smithi155 bash[20805]: cluster 2024-02-18T01:19:46.320836+0000 mgr.y (mgr.24475) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:47.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:47 smithi105 bash[19734]: cluster 2024-02-18T01:19:46.320836+0000 mgr.y (mgr.24475) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:47 smithi105 bash[27669]: cluster 2024-02-18T01:19:46.320836+0000 mgr.y (mgr.24475) 105 : cluster [DBG] pgmap v70: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:48.257 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:48 smithi105 bash[27669]: audit 2024-02-18T01:19:47.374419+0000 mgr.y (mgr.24475) 106 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:48 smithi105 bash[19734]: audit 2024-02-18T01:19:47.374419+0000 mgr.y (mgr.24475) 106 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:48 smithi155 bash[20805]: audit 2024-02-18T01:19:47.374419+0000 mgr.y (mgr.24475) 106 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:49.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:49 smithi105 bash[19734]: cluster 2024-02-18T01:19:48.322328+0000 mgr.y (mgr.24475) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:49.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:49 smithi105 bash[27669]: cluster 2024-02-18T01:19:48.322328+0000 mgr.y (mgr.24475) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:49.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:49 smithi155 bash[20805]: cluster 2024-02-18T01:19:48.322328+0000 mgr.y (mgr.24475) 107 : cluster [DBG] pgmap v71: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:51.215 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- 172.21.15.105:0/409702877 >> v1:172.21.15.105:6790/0 conn(0x7f32c80fc150 legacy=0x7f32c80fc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- 172.21.15.105:0/409702877 shutdown_connections 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- 172.21.15.105:0/409702877 >> 172.21.15.105:0/409702877 conn(0x7f32c806c140 msgr2=0x7f32c806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- 172.21.15.105:0/409702877 shutdown_connections 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- 172.21.15.105:0/409702877 wait complete. 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 Processor -- start 2024-02-18T01:19:51.216 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- start start 2024-02-18T01:19:51.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f32c8114fa0 con 0x7f32c80fef30 2024-02-18T01:19:51.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f32c8116180 con 0x7f32c8102a70 2024-02-18T01:19:51.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32cd359700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f32c8117360 con 0x7f32c80fc150 2024-02-18T01:19:51.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32c6ffd700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f32c80fc150 0x7f32c80fb1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:55088/0 (socket says 172.21.15.105:55088) 2024-02-18T01:19:51.217 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.213+0000 7f32c6ffd700 1 -- 172.21.15.105:0/1769019395 learned_addr learned my addr 172.21.15.105:0/1769019395 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 499351948 0 0) 0x7f32c8117360 con 0x7f32c80fc150 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32b4004500 con 0x7f32c80fc150 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1570022298 0 0) 0x7f32c8114fa0 con 0x7f32c80fef30 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c8117360 con 0x7f32c80fef30 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3361273429 0 0) 0x7f32c8116180 con 0x7f32c8102a70 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32c8114fa0 con 0x7f32c8102a70 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 427599998 0 0) 0x7f32b4004500 con 0x7f32c80fc150 2024-02-18T01:19:51.218 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32c8116180 con 0x7f32c80fc150 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f32b8001370 con 0x7f32c80fc150 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2225280223 0 0) 0x7f32c8117360 con 0x7f32c80fef30 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f32b4004500 con 0x7f32c80fef30 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f32b00019d0 con 0x7f32c80fef30 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2374837863 0 0) 0x7f32c8116180 con 0x7f32c80fc150 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 >> v1:172.21.15.155:6789/0 conn(0x7f32c8102a70 legacy=0x7f32c81137f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.219 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 >> v1:172.21.15.105:6789/0 conn(0x7f32c80fef30 legacy=0x7f32c81100c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32c8118560 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32c8115190 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f32c8115600 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f32b8001590 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f32b8003870 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f32a8000f10 con 0x7f32c80fc150 2024-02-18T01:19:51.220 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.217+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f32b8003ab0 con 0x7f32c80fc150 2024-02-18T01:19:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.221+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f32b80788d0 con 0x7f32c80fc150 2024-02-18T01:19:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.221+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6809/3935238244 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f32b4072980 con 0x7f32b406ec70 2024-02-18T01:19:51.221 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.221+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7f32b8078d00 con 0x7f32c80fc150 2024-02-18T01:19:51.223 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.221+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== osd.2 v1:172.21.15.105:6809/3935238244 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f32b4072980 con 0x7f32b406ec70 2024-02-18T01:19:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 --> v1:172.21.15.105:6809/3935238244 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f32a80029f0 con 0x7f32b406ec70 2024-02-18T01:19:51.240 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32bffff700 1 -- 172.21.15.105:0/1769019395 <== osd.2 v1:172.21.15.105:6809/3935238244 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (unknown 0 0 2379505658) 0x7f32a80029f0 con 0x7f32b406ec70 2024-02-18T01:19:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 >> v1:172.21.15.105:6809/3935238244 conn(0x7f32b406ec70 legacy=0x7f32b4071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 >> v1:172.21.15.105:6800/2148176012 conn(0x7f32b4064430 legacy=0x7f32b40668f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 >> v1:172.21.15.105:6790/0 conn(0x7f32c80fc150 legacy=0x7f32c80fb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:51.241 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32c77fe700 1 -- 172.21.15.105:0/1769019395 reap_dead start 2024-02-18T01:19:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 shutdown_connections 2024-02-18T01:19:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.237+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 >> 172.21.15.105:0/1769019395 conn(0x7f32c806c140 msgr2=0x7f32c8106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.241+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 shutdown_connections 2024-02-18T01:19:51.242 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:51.241+0000 7f32cd359700 1 -- 172.21.15.105:0/1769019395 wait complete. 2024-02-18T01:19:51.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:51 smithi105 bash[19734]: cluster 2024-02-18T01:19:50.323068+0000 mgr.y (mgr.24475) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:51.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:51 smithi105 bash[27669]: cluster 2024-02-18T01:19:50.323068+0000 mgr.y (mgr.24475) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:51.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:51 smithi155 bash[20805]: cluster 2024-02-18T01:19:50.323068+0000 mgr.y (mgr.24475) 108 : cluster [DBG] pgmap v72: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:51.927 INFO:teuthology.orchestra.run.smithi105.stdout:77309411406 2024-02-18T01:19:51.927 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-02-18T01:19:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:19:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:19:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:53 smithi105 bash[27669]: cluster 2024-02-18T01:19:52.324478+0000 mgr.y (mgr.24475) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:53.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:53 smithi105 bash[19734]: cluster 2024-02-18T01:19:52.324478+0000 mgr.y (mgr.24475) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:53.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:53 smithi155 bash[20805]: cluster 2024-02-18T01:19:52.324478+0000 mgr.y (mgr.24475) 109 : cluster [DBG] pgmap v73: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:55 smithi105 bash[19734]: cluster 2024-02-18T01:19:54.325269+0000 mgr.y (mgr.24475) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:55 smithi105 bash[27669]: cluster 2024-02-18T01:19:54.325269+0000 mgr.y (mgr.24475) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:55 smithi155 bash[20805]: cluster 2024-02-18T01:19:54.325269+0000 mgr.y (mgr.24475) 110 : cluster [DBG] pgmap v74: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:56.764 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:19:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:19:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:19:56] "GET /metrics HTTP/1.1" 200 172166 "" "Prometheus/2.33.4" 2024-02-18T01:19:57.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:19:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:19:57.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:57 smithi155 bash[20805]: cluster 2024-02-18T01:19:56.325953+0000 mgr.y (mgr.24475) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:57.782 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:57 smithi105 bash[19734]: cluster 2024-02-18T01:19:56.325953+0000 mgr.y (mgr.24475) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:57.782 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:57 smithi105 bash[27669]: cluster 2024-02-18T01:19:56.325953+0000 mgr.y (mgr.24475) 111 : cluster [DBG] pgmap v75: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:19:58.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:58 smithi105 bash[19734]: audit 2024-02-18T01:19:57.382069+0000 mgr.y (mgr.24475) 112 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:58 smithi105 bash[27669]: audit 2024-02-18T01:19:57.382069+0000 mgr.y (mgr.24475) 112 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:58 smithi155 bash[20805]: audit 2024-02-18T01:19:57.382069+0000 mgr.y (mgr.24475) 112 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:19:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/4037884956 >> v1:172.21.15.105:6790/0 conn(0x7fdec8100890 legacy=0x7fdec8102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/4037884956 shutdown_connections 2024-02-18T01:19:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/4037884956 >> 172.21.15.105:0/4037884956 conn(0x7fdec80f3630 msgr2=0x7fdec80f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/4037884956 shutdown_connections 2024-02-18T01:19:59.822 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/4037884956 wait complete. 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 Processor -- start 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- start start 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdec8117390 con 0x7fdec81043a0 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdec8118590 con 0x7fdec8100890 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fdec8119790 con 0x7fdec80fda40 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdece2d0700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fdec81043a0 0x7fdec8115a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53194/0 (socket says 172.21.15.105:53194) 2024-02-18T01:19:59.823 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdece2d0700 1 -- 172.21.15.105:0/1435820658 learned_addr learned my addr 172.21.15.105:0/1435820658 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3100765217 0 0) 0x7fdec8117390 con 0x7fdec81043a0 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeb4003030 con 0x7fdec81043a0 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1190873876 0 0) 0x7fdec8118590 con 0x7fdec8100890 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8117390 con 0x7fdec8100890 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047834159 0 0) 0x7fdec8119790 con 0x7fdec80fda40 2024-02-18T01:19:59.825 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdec8118590 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4252249167 0 0) 0x7fdeb4003030 con 0x7fdec81043a0 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdeb4003480 con 0x7fdec81043a0 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2020513061 0 0) 0x7fdec8118590 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdeb4003030 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fdec0002cd0 con 0x7fdec81043a0 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fdec40018d0 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3271322570 0 0) 0x7fdec8117390 con 0x7fdec8100890 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fdec8118590 con 0x7fdec8100890 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fdeb8002f20 con 0x7fdec8100890 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2565487742 0 0) 0x7fdeb4003030 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 >> v1:172.21.15.155:6789/0 conn(0x7fdec8100890 legacy=0x7fdec80fa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 >> v1:172.21.15.105:6789/0 conn(0x7fdec81043a0 legacy=0x7fdec8115a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec811a990 con 0x7fdec80fda40 2024-02-18T01:19:59.826 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec8118780 con 0x7fdec80fda40 2024-02-18T01:19:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fdec4002180 con 0x7fdec80fda40 2024-02-18T01:19:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fdec4003100 con 0x7fdec80fda40 2024-02-18T01:19:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.821+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec8118bf0 con 0x7fdec80fda40 2024-02-18T01:19:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.825+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdec8105ab0 con 0x7fdec80fda40 2024-02-18T01:19:59.827 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.825+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fdec4001ce0 con 0x7fdec80fda40 2024-02-18T01:19:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.825+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7fdec4075c10 con 0x7fdec80fda40 2024-02-18T01:19:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.825+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6813/1099653553 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdeb4070ba0 con 0x7fdeb406ce90 2024-02-18T01:19:59.828 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.825+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7fdec4075f90 con 0x7fdec80fda40 2024-02-18T01:19:59.829 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.829+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== osd.3 v1:172.21.15.105:6813/1099653553 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fdeb4070ba0 con 0x7fdeb406ce90 2024-02-18T01:19:59.848 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.845+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 --> v1:172.21.15.105:6813/1099653553 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdec8105e60 con 0x7fdeb406ce90 2024-02-18T01:19:59.849 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.845+0000 7fdebeffd700 1 -- 172.21.15.105:0/1435820658 <== osd.3 v1:172.21.15.105:6813/1099653553 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 1746907681) 0x7fdec8105e60 con 0x7fdeb406ce90 2024-02-18T01:19:59.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 >> v1:172.21.15.105:6813/1099653553 conn(0x7fdeb406ce90 legacy=0x7fdeb406f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 >> v1:172.21.15.105:6800/2148176012 conn(0x7fdeb4062740 legacy=0x7fdeb4064c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 >> v1:172.21.15.105:6790/0 conn(0x7fdec80fda40 legacy=0x7fdec80f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:19:59.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdece2d0700 1 -- 172.21.15.105:0/1435820658 reap_dead start 2024-02-18T01:19:59.850 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 shutdown_connections 2024-02-18T01:19:59.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 >> 172.21.15.105:0/1435820658 conn(0x7fdec80f3630 msgr2=0x7fdec8107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:19:59.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 shutdown_connections 2024-02-18T01:19:59.851 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:19:59.849+0000 7fdecfd33700 1 -- 172.21.15.105:0/1435820658 wait complete. 2024-02-18T01:19:59.932 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:19:59 smithi105 bash[27669]: cluster 2024-02-18T01:19:58.327426+0000 mgr.y (mgr.24475) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:59.933 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:19:59 smithi105 bash[19734]: cluster 2024-02-18T01:19:58.327426+0000 mgr.y (mgr.24475) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:19:59.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:19:59 smithi155 bash[20805]: cluster 2024-02-18T01:19:58.327426+0000 mgr.y (mgr.24475) 113 : cluster [DBG] pgmap v76: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:00.487 INFO:teuthology.orchestra.run.smithi105.stdout:107374182471 2024-02-18T01:20:00.487 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-02-18T01:20:00.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:00 smithi105 bash[19734]: cluster 2024-02-18T01:20:00.000122+0000 mon.a (mon.0) 712 : cluster [INF] overall HEALTH_OK 2024-02-18T01:20:00.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:00 smithi105 bash[27669]: cluster 2024-02-18T01:20:00.000122+0000 mon.a (mon.0) 712 : cluster [INF] overall HEALTH_OK 2024-02-18T01:20:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:00 smithi155 bash[20805]: cluster 2024-02-18T01:20:00.000122+0000 mon.a (mon.0) 712 : cluster [INF] overall HEALTH_OK 2024-02-18T01:20:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:01 smithi155 bash[20805]: cluster 2024-02-18T01:20:00.328333+0000 mgr.y (mgr.24475) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:01 smithi155 bash[20805]: audit 2024-02-18T01:20:00.878144+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:20:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:01 smithi155 bash[20805]: audit 2024-02-18T01:20:01.181422+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[27669]: cluster 2024-02-18T01:20:00.328333+0000 mgr.y (mgr.24475) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[27669]: audit 2024-02-18T01:20:00.878144+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[27669]: audit 2024-02-18T01:20:01.181422+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[19734]: cluster 2024-02-18T01:20:00.328333+0000 mgr.y (mgr.24475) 114 : cluster [DBG] pgmap v77: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[19734]: audit 2024-02-18T01:20:00.878144+0000 mon.a (mon.0) 713 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:20:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:01 smithi105 bash[19734]: audit 2024-02-18T01:20:01.181422+0000 mon.a (mon.0) 714 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:03.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:03 smithi105 bash[19734]: cluster 2024-02-18T01:20:02.329720+0000 mgr.y (mgr.24475) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:03.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:03 smithi105 bash[27669]: cluster 2024-02-18T01:20:02.329720+0000 mgr.y (mgr.24475) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:03.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:03 smithi155 bash[20805]: cluster 2024-02-18T01:20:02.329720+0000 mgr.y (mgr.24475) 115 : cluster [DBG] pgmap v78: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:05.333 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:05.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:05 smithi105 bash[27669]: cluster 2024-02-18T01:20:04.330426+0000 mgr.y (mgr.24475) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:05.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:05 smithi105 bash[19734]: cluster 2024-02-18T01:20:04.330426+0000 mgr.y (mgr.24475) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:05.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:05 smithi155 bash[20805]: cluster 2024-02-18T01:20:04.330426+0000 mgr.y (mgr.24475) 116 : cluster [DBG] pgmap v79: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:06] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:20:07.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[20805]: cluster 2024-02-18T01:20:06.330860+0000 mgr.y (mgr.24475) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[20805]: audit 2024-02-18T01:20:07.227421+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[20805]: audit 2024-02-18T01:20:07.521508+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:20:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[20805]: audit 2024-02-18T01:20:07.523395+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:20:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:07 smithi155 bash[20805]: audit 2024-02-18T01:20:07.534816+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:07.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[19734]: cluster 2024-02-18T01:20:06.330860+0000 mgr.y (mgr.24475) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:07.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[19734]: audit 2024-02-18T01:20:07.227421+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[19734]: audit 2024-02-18T01:20:07.521508+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[19734]: audit 2024-02-18T01:20:07.523395+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[19734]: audit 2024-02-18T01:20:07.534816+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[27669]: cluster 2024-02-18T01:20:06.330860+0000 mgr.y (mgr.24475) 117 : cluster [DBG] pgmap v80: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[27669]: audit 2024-02-18T01:20:07.227421+0000 mon.a (mon.0) 715 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[27669]: audit 2024-02-18T01:20:07.521508+0000 mon.a (mon.0) 716 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:20:07.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[27669]: audit 2024-02-18T01:20:07.523395+0000 mon.a (mon.0) 717 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:20:07.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:07 smithi105 bash[27669]: audit 2024-02-18T01:20:07.534816+0000 mon.a (mon.0) 718 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:20:08.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/1855648579 >> v1:172.21.15.105:6790/0 conn(0x7fcfa0100890 legacy=0x7fcfa0102c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/1855648579 shutdown_connections 2024-02-18T01:20:08.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/1855648579 >> 172.21.15.105:0/1855648579 conn(0x7fcfa00f36b0 msgr2=0x7fcfa00f5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:08.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/1855648579 shutdown_connections 2024-02-18T01:20:08.406 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/1855648579 wait complete. 2024-02-18T01:20:08.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 Processor -- start 2024-02-18T01:20:08.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- start start 2024-02-18T01:20:08.407 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcfa00faa60 con 0x7fcfa00fda40 2024-02-18T01:20:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcfa01188c0 con 0x7fcfa01043a0 2024-02-18T01:20:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcfa0119aa0 con 0x7fcfa0100890 2024-02-18T01:20:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa554f700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fcfa0100890 0x7fcfa0112a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:41074/0 (socket says 172.21.15.105:41074) 2024-02-18T01:20:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa554f700 1 -- 172.21.15.105:0/320478869 learned_addr learned my addr 172.21.15.105:0/320478869 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:08.408 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa5d50700 1 --1- 172.21.15.105:0/320478869 >> v1:172.21.15.105:6789/0 conn(0x7fcfa00fda40 0x7fcfa00f9e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:53212/0 (socket says 172.21.15.105:53212) 2024-02-18T01:20:08.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1362828959 0 0) 0x7fcfa0119aa0 con 0x7fcfa0100890 2024-02-18T01:20:08.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf8c003030 con 0x7fcfa0100890 2024-02-18T01:20:08.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2245046519 0 0) 0x7fcfa00faa60 con 0x7fcfa00fda40 2024-02-18T01:20:08.409 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfa0119aa0 con 0x7fcfa00fda40 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3632285643 0 0) 0x7fcf8c003030 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcfa00faa60 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fcf900015d0 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3225507948 0 0) 0x7fcfa01188c0 con 0x7fcfa01043a0 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf8c003030 con 0x7fcfa01043a0 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 264828542 0 0) 0x7fcfa00faa60 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 >> v1:172.21.15.155:6789/0 conn(0x7fcfa01043a0 legacy=0x7fcfa0116190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 >> v1:172.21.15.105:6789/0 conn(0x7fcfa00fda40 legacy=0x7fcfa00f9e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa011ac80 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfa0117910 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfa0117d80 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.404+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fcf90000fe0 con 0x7fcfa0100890 2024-02-18T01:20:08.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fcf90003960 con 0x7fcfa0100890 2024-02-18T01:20:08.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcfa00ffbd0 con 0x7fcfa0100890 2024-02-18T01:20:08.411 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fcf90003c40 con 0x7fcfa0100890 2024-02-18T01:20:08.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7fcf90078a90 con 0x7fcfa0100890 2024-02-18T01:20:08.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.155:6800/1608438726 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcf8c0704c0 con 0x7fcf8c06c850 2024-02-18T01:20:08.413 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.408+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7fcf90078e60 con 0x7fcfa0100890 2024-02-18T01:20:08.415 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.412+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== osd.4 v1:172.21.15.155:6800/1608438726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7fcf8c0704c0 con 0x7fcf8c06c850 2024-02-18T01:20:08.434 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 --> v1:172.21.15.155:6800/1608438726 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcfa00fff80 con 0x7fcf8c06c850 2024-02-18T01:20:08.435 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcf96ffd700 1 -- 172.21.15.105:0/320478869 <== osd.4 v1:172.21.15.155:6800/1608438726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3492131060) 0x7fcfa00fff80 con 0x7fcf8c06c850 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 >> v1:172.21.15.155:6800/1608438726 conn(0x7fcf8c06c850 legacy=0x7fcf8c06ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 >> v1:172.21.15.105:6800/2148176012 conn(0x7fcf8c0620a0 legacy=0x7fcf8c064560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 >> v1:172.21.15.105:6790/0 conn(0x7fcfa0100890 legacy=0x7fcfa0112a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa6551700 1 -- 172.21.15.105:0/320478869 reap_dead start 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 shutdown_connections 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 >> 172.21.15.105:0/320478869 conn(0x7fcfa00f36b0 msgr2=0x7fcfa0107800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 shutdown_connections 2024-02-18T01:20:08.436 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:08.432+0000 7fcfa7fb4700 1 -- 172.21.15.105:0/320478869 wait complete. 2024-02-18T01:20:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:08 smithi105 bash[19734]: audit 2024-02-18T01:20:07.390056+0000 mgr.y (mgr.24475) 118 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:08 smithi105 bash[27669]: audit 2024-02-18T01:20:07.390056+0000 mgr.y (mgr.24475) 118 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:08 smithi155 bash[20805]: audit 2024-02-18T01:20:07.390056+0000 mgr.y (mgr.24475) 118 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:09.068 INFO:teuthology.orchestra.run.smithi105.stdout:133143986240 2024-02-18T01:20:09.069 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-02-18T01:20:09.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:09 smithi105 bash[19734]: cluster 2024-02-18T01:20:08.331924+0000 mgr.y (mgr.24475) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:09 smithi105 bash[27669]: cluster 2024-02-18T01:20:08.331924+0000 mgr.y (mgr.24475) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:09 smithi155 bash[20805]: cluster 2024-02-18T01:20:08.331924+0000 mgr.y (mgr.24475) 119 : cluster [DBG] pgmap v81: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:11 smithi155 bash[20805]: cluster 2024-02-18T01:20:10.332601+0000 mgr.y (mgr.24475) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:11.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:11 smithi105 bash[19734]: cluster 2024-02-18T01:20:10.332601+0000 mgr.y (mgr.24475) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:11.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:11 smithi105 bash[27669]: cluster 2024-02-18T01:20:10.332601+0000 mgr.y (mgr.24475) 120 : cluster [DBG] pgmap v82: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:13.909 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:13.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:13 smithi105 bash[27669]: cluster 2024-02-18T01:20:12.334003+0000 mgr.y (mgr.24475) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:13 smithi105 bash[19734]: cluster 2024-02-18T01:20:12.334003+0000 mgr.y (mgr.24475) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:13 smithi155 bash[20805]: cluster 2024-02-18T01:20:12.334003+0000 mgr.y (mgr.24475) 121 : cluster [DBG] pgmap v83: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:15 smithi105 bash[19734]: cluster 2024-02-18T01:20:14.334711+0000 mgr.y (mgr.24475) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:15 smithi105 bash[27669]: cluster 2024-02-18T01:20:14.334711+0000 mgr.y (mgr.24475) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:15 smithi155 bash[20805]: cluster 2024-02-18T01:20:14.334711+0000 mgr.y (mgr.24475) 122 : cluster [DBG] pgmap v84: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/3736261012 >> v1:172.21.15.105:6789/0 conn(0x7f88e41043d0 legacy=0x7f88e4106870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/3736261012 shutdown_connections 2024-02-18T01:20:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/3736261012 >> 172.21.15.105:0/3736261012 conn(0x7f88e406c140 msgr2=0x7f88e406c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:16.954 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/3736261012 shutdown_connections 2024-02-18T01:20:16.955 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/3736261012 wait complete. 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 Processor -- start 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- start start 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88e40f8eb0 con 0x7f88e40fdae0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88e40f9040 con 0x7f88e41043d0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f88e40f9210 con 0x7f88e41008c0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88db7fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f88e41008c0 0x7f88e4114e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:41632/0 (socket says 172.21.15.105:41632) 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e3fff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f88e40fdae0 0x7f88e40fcb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:59020/0 (socket says 172.21.15.105:59020) 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88db7fe700 1 -- 172.21.15.105:0/2480975643 learned_addr learned my addr 172.21.15.105:0/2480975643 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 618895429 0 0) 0x7f88e40f9210 con 0x7f88e41008c0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d00033f0 con 0x7f88e41008c0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4209170921 0 0) 0x7f88e40f8eb0 con 0x7f88e40fdae0 2024-02-18T01:20:16.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88e40f9210 con 0x7f88e40fdae0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1263260979 0 0) 0x7f88d00033f0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88e40f8eb0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f88cc0018e0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 705124913 0 0) 0x7f88e40f9040 con 0x7f88e41043d0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88d00033f0 con 0x7f88e41043d0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1376738990 0 0) 0x7f88e40f9210 con 0x7f88e40fdae0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f88e40f9040 con 0x7f88e40fdae0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f88d4001880 con 0x7f88e40fdae0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4283496212 0 0) 0x7f88e40f8eb0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 >> v1:172.21.15.155:6789/0 conn(0x7f88e41043d0 legacy=0x7f88e4118550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 >> v1:172.21.15.105:6789/0 conn(0x7f88e40fdae0 legacy=0x7f88e40fcb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e411cca0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e4119cd0 con 0x7f88e41008c0 2024-02-18T01:20:16.958 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e411a1c0 con 0x7f88e41008c0 2024-02-18T01:20:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f88cc001fa0 con 0x7f88e41008c0 2024-02-18T01:20:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.952+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f88cc0059e0 con 0x7f88e41008c0 2024-02-18T01:20:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.956+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f88c4000f10 con 0x7f88e41008c0 2024-02-18T01:20:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.956+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f88cc001b00 con 0x7f88e41008c0 2024-02-18T01:20:16.959 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.956+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f88cc078510 con 0x7f88e41008c0 2024-02-18T01:20:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.956+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.155:6804/3674587623 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f88d0070800 con 0x7f88d006cb90 2024-02-18T01:20:16.960 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.956+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7f88cc0788e0 con 0x7f88e41008c0 2024-02-18T01:20:16.962 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.960+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== osd.5 v1:172.21.15.155:6804/3674587623 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f88d0070800 con 0x7f88d006cb90 2024-02-18T01:20:16.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 --> v1:172.21.15.155:6804/3674587623 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f88c40029f0 con 0x7f88d006cb90 2024-02-18T01:20:16.983 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88e1ffb700 1 -- 172.21.15.105:0/2480975643 <== osd.5 v1:172.21.15.155:6804/3674587623 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 2227658367) 0x7f88c40029f0 con 0x7f88d006cb90 2024-02-18T01:20:16.984 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 >> v1:172.21.15.155:6804/3674587623 conn(0x7f88d006cb90 legacy=0x7f88d006f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 >> v1:172.21.15.105:6800/2148176012 conn(0x7f88d0062380 legacy=0x7f88d0064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 >> v1:172.21.15.105:6790/0 conn(0x7f88e41008c0 legacy=0x7f88e4114e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88e8b79700 1 -- 172.21.15.105:0/2480975643 reap_dead start 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 shutdown_connections 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 >> 172.21.15.105:0/2480975643 conn(0x7f88e406c140 msgr2=0x7f88e4106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 shutdown_connections 2024-02-18T01:20:16.988 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:16.980+0000 7f88ea5dc700 1 -- 172.21.15.105:0/2480975643 wait complete. 2024-02-18T01:20:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:16] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:20:17.620 INFO:teuthology.orchestra.run.smithi105.stdout:158913790010 2024-02-18T01:20:17.620 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-02-18T01:20:17.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:17.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:17 smithi155 bash[20805]: cluster 2024-02-18T01:20:16.335439+0000 mgr.y (mgr.24475) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:17.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:17 smithi105 bash[19734]: cluster 2024-02-18T01:20:16.335439+0000 mgr.y (mgr.24475) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:17.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:17 smithi105 bash[27669]: cluster 2024-02-18T01:20:16.335439+0000 mgr.y (mgr.24475) 123 : cluster [DBG] pgmap v85: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:18.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:18 smithi105 bash[19734]: audit 2024-02-18T01:20:17.401168+0000 mgr.y (mgr.24475) 124 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:18.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:18 smithi105 bash[27669]: audit 2024-02-18T01:20:17.401168+0000 mgr.y (mgr.24475) 124 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:18 smithi155 bash[20805]: audit 2024-02-18T01:20:17.401168+0000 mgr.y (mgr.24475) 124 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:19 smithi105 bash[27669]: cluster 2024-02-18T01:20:18.336856+0000 mgr.y (mgr.24475) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:19.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:19 smithi105 bash[19734]: cluster 2024-02-18T01:20:18.336856+0000 mgr.y (mgr.24475) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:19.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:19 smithi155 bash[20805]: cluster 2024-02-18T01:20:18.336856+0000 mgr.y (mgr.24475) 125 : cluster [DBG] pgmap v86: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:21.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:21 smithi155 bash[20805]: cluster 2024-02-18T01:20:20.337574+0000 mgr.y (mgr.24475) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:21.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:21 smithi105 bash[19734]: cluster 2024-02-18T01:20:20.337574+0000 mgr.y (mgr.24475) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:21.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:21 smithi105 bash[27669]: cluster 2024-02-18T01:20:20.337574+0000 mgr.y (mgr.24475) 126 : cluster [DBG] pgmap v87: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:22.448 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:23.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:23 smithi105 bash[27669]: cluster 2024-02-18T01:20:22.338979+0000 mgr.y (mgr.24475) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:23.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:23 smithi105 bash[19734]: cluster 2024-02-18T01:20:22.338979+0000 mgr.y (mgr.24475) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:23 smithi155 bash[20805]: cluster 2024-02-18T01:20:22.338979+0000 mgr.y (mgr.24475) 127 : cluster [DBG] pgmap v88: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- 172.21.15.105:0/3119992229 >> v1:172.21.15.105:6789/0 conn(0x7f48b80f95f0 legacy=0x7f48b8102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- 172.21.15.105:0/3119992229 shutdown_connections 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- 172.21.15.105:0/3119992229 >> 172.21.15.105:0/3119992229 conn(0x7f48b80f2570 msgr2=0x7f48b80f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- 172.21.15.105:0/3119992229 shutdown_connections 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- 172.21.15.105:0/3119992229 wait complete. 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 Processor -- start 2024-02-18T01:20:25.295 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- start start 2024-02-18T01:20:25.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f48b8117520 con 0x7f48b81043d0 2024-02-18T01:20:25.296 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f48b8118700 con 0x7f48b80f95f0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48be1db700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f48b8119900 con 0x7f48b80f8a40 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b77fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f48b80f8a40 0x7f48b80ff810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:52998/0 (socket says 172.21.15.105:52998) 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b77fe700 1 -- 172.21.15.105:0/2518416199 learned_addr learned my addr 172.21.15.105:0/2518416199 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1808295541 0 0) 0x7f48b8117520 con 0x7f48b81043d0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48a40033f0 con 0x7f48b81043d0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3591821979 0 0) 0x7f48b8119900 con 0x7f48b80f8a40 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8117520 con 0x7f48b80f8a40 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347109353 0 0) 0x7f48b8118700 con 0x7f48b80f95f0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48b8119900 con 0x7f48b80f95f0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1454832100 0 0) 0x7f48a40033f0 con 0x7f48b81043d0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48b8118700 con 0x7f48b81043d0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f48b0001840 con 0x7f48b81043d0 2024-02-18T01:20:25.297 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 486867184 0 0) 0x7f48b8117520 con 0x7f48b80f8a40 2024-02-18T01:20:25.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48a40033f0 con 0x7f48b80f8a40 2024-02-18T01:20:25.298 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f48a8002d50 con 0x7f48b80f8a40 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3124177429 0 0) 0x7f48b8119900 con 0x7f48b80f95f0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f48b8117520 con 0x7f48b80f95f0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1849240881 0 0) 0x7f48b8118700 con 0x7f48b81043d0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 >> v1:172.21.15.105:6790/0 conn(0x7f48b80f8a40 legacy=0x7f48b80ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 >> v1:172.21.15.155:6789/0 conn(0x7f48b80f95f0 legacy=0x7f48b80fff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.292+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b811ab00 con 0x7f48b81043d0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48b8117710 con 0x7f48b81043d0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f48b8117b80 con 0x7f48b81043d0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f48b0001ec0 con 0x7f48b81043d0 2024-02-18T01:20:25.299 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f48b0003180 con 0x7f48b81043d0 2024-02-18T01:20:25.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.105:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4898000f10 con 0x7f48b81043d0 2024-02-18T01:20:25.300 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.296+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f48b0004410 con 0x7f48b81043d0 2024-02-18T01:20:25.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.300+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f48b0075bc0 con 0x7f48b81043d0 2024-02-18T01:20:25.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.300+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.155:6808/98819544 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f48a4074be0 con 0x7f48a4070ed0 2024-02-18T01:20:25.302 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.300+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7f48b0075f90 con 0x7f48b81043d0 2024-02-18T01:20:25.304 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.300+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== osd.6 v1:172.21.15.155:6808/98819544 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f48a4074be0 con 0x7f48a4070ed0 2024-02-18T01:20:25.323 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 --> v1:172.21.15.155:6808/98819544 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f48980029f0 con 0x7f48a4070ed0 2024-02-18T01:20:25.324 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48b4ff9700 1 -- 172.21.15.105:0/2518416199 <== osd.6 v1:172.21.15.155:6808/98819544 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3151770417) 0x7f48980029f0 con 0x7f48a4070ed0 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 >> v1:172.21.15.155:6808/98819544 conn(0x7f48a4070ed0 legacy=0x7f48a4073370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 >> v1:172.21.15.105:6800/2148176012 conn(0x7f48a4066690 legacy=0x7f48a4068b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 >> v1:172.21.15.105:6789/0 conn(0x7f48b81043d0 legacy=0x7f48b8115be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48b7fff700 1 -- 172.21.15.105:0/2518416199 reap_dead start 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 shutdown_connections 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 >> 172.21.15.105:0/2518416199 conn(0x7f48b80f2570 msgr2=0x7f48b8107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 shutdown_connections 2024-02-18T01:20:25.325 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:25.320+0000 7f48be1db700 1 -- 172.21.15.105:0/2518416199 wait complete. 2024-02-18T01:20:25.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:25 smithi105 bash[27669]: cluster 2024-02-18T01:20:24.339469+0000 mgr.y (mgr.24475) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:25 smithi105 bash[19734]: cluster 2024-02-18T01:20:24.339469+0000 mgr.y (mgr.24475) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:25.986 INFO:teuthology.orchestra.run.smithi105.stdout:184683593780 2024-02-18T01:20:25.986 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-02-18T01:20:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:25 smithi155 bash[20805]: cluster 2024-02-18T01:20:24.339469+0000 mgr.y (mgr.24475) 128 : cluster [DBG] pgmap v89: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:26] "GET /metrics HTTP/1.1" 200 172165 "" "Prometheus/2.33.4" 2024-02-18T01:20:27.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:27 smithi155 bash[20805]: cluster 2024-02-18T01:20:26.340248+0000 mgr.y (mgr.24475) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:27.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:27.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:27 smithi105 bash[19734]: cluster 2024-02-18T01:20:26.340248+0000 mgr.y (mgr.24475) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:27 smithi105 bash[27669]: cluster 2024-02-18T01:20:26.340248+0000 mgr.y (mgr.24475) 129 : cluster [DBG] pgmap v90: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:28 smithi105 bash[19734]: audit 2024-02-18T01:20:27.412512+0000 mgr.y (mgr.24475) 130 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:28 smithi105 bash[27669]: audit 2024-02-18T01:20:27.412512+0000 mgr.y (mgr.24475) 130 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:28 smithi155 bash[20805]: audit 2024-02-18T01:20:27.412512+0000 mgr.y (mgr.24475) 130 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:29 smithi105 bash[27669]: cluster 2024-02-18T01:20:28.341757+0000 mgr.y (mgr.24475) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:29 smithi105 bash[19734]: cluster 2024-02-18T01:20:28.341757+0000 mgr.y (mgr.24475) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:29.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:29 smithi155 bash[20805]: cluster 2024-02-18T01:20:28.341757+0000 mgr.y (mgr.24475) 131 : cluster [DBG] pgmap v91: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:30.830 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:31.984 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:31 smithi105 bash[19734]: cluster 2024-02-18T01:20:30.342450+0000 mgr.y (mgr.24475) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:31.984 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:31 smithi105 bash[27669]: cluster 2024-02-18T01:20:30.342450+0000 mgr.y (mgr.24475) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:31.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:31 smithi155 bash[20805]: cluster 2024-02-18T01:20:30.342450+0000 mgr.y (mgr.24475) 132 : cluster [DBG] pgmap v92: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:31.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:33.640 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.636+0000 7f9528a83700 1 -- 172.21.15.105:0/1719991190 >> v1:172.21.15.105:6790/0 conn(0x7f9524102b40 legacy=0x7f9524104f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.636+0000 7f9528a83700 1 -- 172.21.15.105:0/1719991190 shutdown_connections 2024-02-18T01:20:33.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.636+0000 7f9528a83700 1 -- 172.21.15.105:0/1719991190 >> 172.21.15.105:0/1719991190 conn(0x7f95240f5890 msgr2=0x7f95240f7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:33.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.636+0000 7f9528a83700 1 -- 172.21.15.105:0/1719991190 shutdown_connections 2024-02-18T01:20:33.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.636+0000 7f9528a83700 1 -- 172.21.15.105:0/1719991190 wait complete. 2024-02-18T01:20:33.641 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 Processor -- start 2024-02-18T01:20:33.642 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- start start 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f9524119660 con 0x7f9524106650 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9522d9d700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f9524106650 0x7f9524117d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:54130/0 (socket says 172.21.15.105:54130) 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9522d9d700 1 -- 172.21.15.105:0/734218400 learned_addr learned my addr 172.21.15.105:0/734218400 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9521d9b700 1 --1- 172.21.15.105:0/734218400 >> v1:172.21.15.105:6790/0 conn(0x7f9524102b40 0x7f95240fc5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:35542/0 (socket says 172.21.15.105:35542) 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f952411a840 con 0x7f95240ffc70 2024-02-18T01:20:33.643 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f952411ba40 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 912270108 0 0) 0x7f952411ba40 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9510003630 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224986526 0 0) 0x7f9524119660 con 0x7f9524106650 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f952411ba40 con 0x7f9524106650 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1339645437 0 0) 0x7f952411a840 con 0x7f95240ffc70 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9524119660 con 0x7f95240ffc70 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1492971087 0 0) 0x7f9510003630 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f952411a840 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3452474038 0 0) 0x7f952411ba40 con 0x7f9524106650 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f9510003630 con 0x7f9524106650 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f950c001f90 con 0x7f9524102b40 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f951c001bc0 con 0x7f9524106650 2024-02-18T01:20:33.645 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 684378775 0 0) 0x7f9524119660 con 0x7f95240ffc70 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f952411ba40 con 0x7f95240ffc70 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f95140018e0 con 0x7f95240ffc70 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2556148106 0 0) 0x7f952411a840 con 0x7f9524102b40 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 >> v1:172.21.15.155:6789/0 conn(0x7f95240ffc70 legacy=0x7f95240fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 >> v1:172.21.15.105:6789/0 conn(0x7f9524106650 legacy=0x7f9524117d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952411cc40 con 0x7f9524102b40 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952411aa30 con 0x7f9524102b40 2024-02-18T01:20:33.646 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f952411af20 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f950c002c20 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.105:6790/0 -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9524101ef0 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.640+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f950c003720 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.644+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f950c003960 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.644+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f950c078840 con 0x7f9524102b40 2024-02-18T01:20:33.647 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.644+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.155:6812/1643073216 -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9510070ad0 con 0x7f951006ce60 2024-02-18T01:20:33.648 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.644+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (unknown 2884312405 0 0) 0x7f950c078c70 con 0x7f9524102b40 2024-02-18T01:20:33.650 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.648+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== osd.7 v1:172.21.15.155:6812/1643073216 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (unknown 0 0 1671252174) 0x7f9510070ad0 con 0x7f951006ce60 2024-02-18T01:20:33.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 --> v1:172.21.15.155:6812/1643073216 -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f95241022a0 con 0x7f951006ce60 2024-02-18T01:20:33.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f951b7fe700 1 -- 172.21.15.105:0/734218400 <== osd.7 v1:172.21.15.155:6812/1643073216 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (unknown 0 0 3254715640) 0x7f95241022a0 con 0x7f951006ce60 2024-02-18T01:20:33.673 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 >> v1:172.21.15.155:6812/1643073216 conn(0x7f951006ce60 legacy=0x7f951006f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 >> v1:172.21.15.105:6800/2148176012 conn(0x7f9510062590 legacy=0x7f9510064a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 >> v1:172.21.15.105:6790/0 conn(0x7f9524102b40 legacy=0x7f95240fc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.668+0000 7f9522d9d700 1 -- 172.21.15.105:0/734218400 reap_dead start 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.672+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 shutdown_connections 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.672+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 >> 172.21.15.105:0/734218400 conn(0x7f95240f5890 msgr2=0x7f9524109a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.672+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 shutdown_connections 2024-02-18T01:20:33.674 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:33.672+0000 7f9528a83700 1 -- 172.21.15.105:0/734218400 wait complete. 2024-02-18T01:20:33.731 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:33 smithi105 bash[27669]: cluster 2024-02-18T01:20:32.343268+0000 mgr.y (mgr.24475) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:33.731 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:33 smithi105 bash[19734]: cluster 2024-02-18T01:20:32.343268+0000 mgr.y (mgr.24475) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:33 smithi155 bash[20805]: cluster 2024-02-18T01:20:32.343268+0000 mgr.y (mgr.24475) 133 : cluster [DBG] pgmap v93: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:34.345 INFO:teuthology.orchestra.run.smithi105.stdout:210453397549 2024-02-18T01:20:34.346 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-02-18T01:20:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:35 smithi105 bash[27669]: cluster 2024-02-18T01:20:34.344052+0000 mgr.y (mgr.24475) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:35 smithi105 bash[19734]: cluster 2024-02-18T01:20:34.344052+0000 mgr.y (mgr.24475) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:35 smithi155 bash[20805]: cluster 2024-02-18T01:20:34.344052+0000 mgr.y (mgr.24475) 134 : cluster [DBG] pgmap v94: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:36] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:20:37.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:37 smithi155 bash[20805]: cluster 2024-02-18T01:20:36.344834+0000 mgr.y (mgr.24475) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:37.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:37 smithi105 bash[19734]: cluster 2024-02-18T01:20:36.344834+0000 mgr.y (mgr.24475) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:37 smithi105 bash[27669]: cluster 2024-02-18T01:20:36.344834+0000 mgr.y (mgr.24475) 135 : cluster [DBG] pgmap v95: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:20:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:38 smithi105 bash[19734]: audit 2024-02-18T01:20:37.423929+0000 mgr.y (mgr.24475) 136 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:38 smithi105 bash[27669]: audit 2024-02-18T01:20:37.423929+0000 mgr.y (mgr.24475) 136 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:38 smithi155 bash[20805]: audit 2024-02-18T01:20:37.423929+0000 mgr.y (mgr.24475) 136 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:39.177 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:39 smithi105 bash[19734]: cluster 2024-02-18T01:20:38.346339+0000 mgr.y (mgr.24475) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:39 smithi105 bash[27669]: cluster 2024-02-18T01:20:38.346339+0000 mgr.y (mgr.24475) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:39 smithi155 bash[20805]: cluster 2024-02-18T01:20:38.346339+0000 mgr.y (mgr.24475) 137 : cluster [DBG] pgmap v96: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:41.966 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:41 smithi105 bash[27669]: cluster 2024-02-18T01:20:40.347220+0000 mgr.y (mgr.24475) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:41.966 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:41 smithi105 bash[19734]: cluster 2024-02-18T01:20:40.347220+0000 mgr.y (mgr.24475) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:41.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:41 smithi155 bash[20805]: cluster 2024-02-18T01:20:40.347220+0000 mgr.y (mgr.24475) 138 : cluster [DBG] pgmap v97: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:41.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:42.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.104+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3621706099 >> v1:172.21.15.105:6789/0 conn(0x7f2b240f6870 legacy=0x7f2b240f6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:42.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.104+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3621706099 shutdown_connections 2024-02-18T01:20:42.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.104+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3621706099 >> 172.21.15.105:0/3621706099 conn(0x7f2b240f25d0 msgr2=0x7f2b240f49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:42.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.104+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3621706099 shutdown_connections 2024-02-18T01:20:42.110 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.104+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3621706099 wait complete. 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 Processor -- start 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- start start 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2b2411d900 con 0x7f2b240f7420 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2b2411eae0 con 0x7f2b240f6870 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2b2411fce0 con 0x7f2b2410ade0 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b29847700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f2b2410ade0 0x7f2b2411c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:57726/0 (socket says 172.21.15.105:57726) 2024-02-18T01:20:42.111 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b29847700 1 -- 172.21.15.105:0/3261927521 learned_addr learned my addr 172.21.15.105:0/3261927521 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:42.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395487435 0 0) 0x7f2b2411d900 con 0x7f2b240f7420 2024-02-18T01:20:42.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b100033f0 con 0x7f2b240f7420 2024-02-18T01:20:42.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 515617562 0 0) 0x7f2b2411fce0 con 0x7f2b2410ade0 2024-02-18T01:20:42.113 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2411d900 con 0x7f2b2410ade0 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2231736702 0 0) 0x7f2b2411eae0 con 0x7f2b240f6870 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b2411fce0 con 0x7f2b240f6870 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3097005004 0 0) 0x7f2b100033f0 con 0x7f2b240f7420 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b2411eae0 con 0x7f2b240f7420 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2b0c001cd0 con 0x7f2b240f7420 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2613015999 0 0) 0x7f2b2411d900 con 0x7f2b2410ade0 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b100033f0 con 0x7f2b2410ade0 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2b1c0018c0 con 0x7f2b2410ade0 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1110496367 0 0) 0x7f2b2411fce0 con 0x7f2b240f6870 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f2b2411d900 con 0x7f2b240f6870 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 861132742 0 0) 0x7f2b2411eae0 con 0x7f2b240f7420 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 >> v1:172.21.15.105:6790/0 conn(0x7f2b2410ade0 legacy=0x7f2b2411c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 >> v1:172.21.15.155:6789/0 conn(0x7f2b240f6870 legacy=0x7f2b24106f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:42.114 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b24120ee0 con 0x7f2b240f7420 2024-02-18T01:20:42.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2411fed0 con 0x7f2b240f7420 2024-02-18T01:20:42.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b24120340 con 0x7f2b240f7420 2024-02-18T01:20:42.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f2b0c002720 con 0x7f2b240f7420 2024-02-18T01:20:42.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.108+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2b0c003570 con 0x7f2b240f7420 2024-02-18T01:20:42.115 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.112+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b04004f20 con 0x7f2b240f7420 2024-02-18T01:20:42.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.112+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f2b0c002280 con 0x7f2b240f7420 2024-02-18T01:20:42.116 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.112+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f2b0c075fe0 con 0x7f2b240f7420 2024-02-18T01:20:42.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.116+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f2b0c049b70 con 0x7f2b240f7420 2024-02-18T01:20:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.296+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b040051d0 con 0x7f2b240f7420 2024-02-18T01:20:42.301 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.296+0000 7f2b22777700 1 -- 172.21.15.105:0/3261927521 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (unknown 574334944 0 70133040) 0x7f2b0c047fe0 con 0x7f2b240f7420 2024-02-18T01:20:42.301 INFO:teuthology.orchestra.run.smithi105.stdout:34359738472 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 >> v1:172.21.15.105:6800/2148176012 conn(0x7f2b10062460 legacy=0x7f2b10064920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 >> v1:172.21.15.105:6789/0 conn(0x7f2b240f7420 legacy=0x7f2b241076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 shutdown_connections 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 >> 172.21.15.105:0/3261927521 conn(0x7f2b240f25d0 msgr2=0x7f2b240f3b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 shutdown_connections 2024-02-18T01:20:42.303 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:42.300+0000 7f2b2b2aa700 1 -- 172.21.15.105:0/3261927521 wait complete. 2024-02-18T01:20:42.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738459 got 34359738472 for osd.0 2024-02-18T01:20:42.905 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-02-18T01:20:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:42 smithi105 bash[19734]: audit 2024-02-18T01:20:42.300512+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? v1:172.21.15.105:0/3261927521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T01:20:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:42 smithi105 bash[27669]: audit 2024-02-18T01:20:42.300512+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? v1:172.21.15.105:0/3261927521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T01:20:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:42 smithi155 bash[20805]: audit 2024-02-18T01:20:42.300512+0000 mon.a (mon.0) 719 : audit [DBG] from='client.? v1:172.21.15.105:0/3261927521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-02-18T01:20:43.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:43 smithi105 bash[19734]: cluster 2024-02-18T01:20:42.348873+0000 mgr.y (mgr.24475) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:43 smithi105 bash[27669]: cluster 2024-02-18T01:20:42.348873+0000 mgr.y (mgr.24475) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:43 smithi155 bash[20805]: cluster 2024-02-18T01:20:42.348873+0000 mgr.y (mgr.24475) 139 : cluster [DBG] pgmap v98: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:45 smithi105 bash[19734]: cluster 2024-02-18T01:20:44.349641+0000 mgr.y (mgr.24475) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:45 smithi105 bash[27669]: cluster 2024-02-18T01:20:44.349641+0000 mgr.y (mgr.24475) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:45 smithi155 bash[20805]: cluster 2024-02-18T01:20:44.349641+0000 mgr.y (mgr.24475) 140 : cluster [DBG] pgmap v99: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:46] "GET /metrics HTTP/1.1" 200 172159 "" "Prometheus/2.33.4" 2024-02-18T01:20:47.713 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:47.747 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:47 smithi105 bash[19734]: cluster 2024-02-18T01:20:46.350348+0000 mgr.y (mgr.24475) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:20:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:47 smithi105 bash[27669]: cluster 2024-02-18T01:20:46.350348+0000 mgr.y (mgr.24475) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:20:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:47 smithi155 bash[20805]: cluster 2024-02-18T01:20:46.350348+0000 mgr.y (mgr.24475) 141 : cluster [DBG] pgmap v100: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:20:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:48 smithi105 bash[19734]: audit 2024-02-18T01:20:47.435201+0000 mgr.y (mgr.24475) 142 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:48 smithi105 bash[27669]: audit 2024-02-18T01:20:47.435201+0000 mgr.y (mgr.24475) 142 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:48 smithi155 bash[20805]: audit 2024-02-18T01:20:47.435201+0000 mgr.y (mgr.24475) 142 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:49 smithi105 bash[19734]: cluster 2024-02-18T01:20:48.351910+0000 mgr.y (mgr.24475) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:20:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:49 smithi105 bash[27669]: cluster 2024-02-18T01:20:48.351910+0000 mgr.y (mgr.24475) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:20:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:49 smithi155 bash[20805]: cluster 2024-02-18T01:20:48.351910+0000 mgr.y (mgr.24475) 143 : cluster [DBG] pgmap v101: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:20:50.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.779+0000 7f81c095e700 1 -- 172.21.15.105:0/1465894421 >> v1:172.21.15.105:6790/0 conn(0x7f81bc1043b0 legacy=0x7f81bc106850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:50.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.779+0000 7f81c095e700 1 -- 172.21.15.105:0/1465894421 shutdown_connections 2024-02-18T01:20:50.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.779+0000 7f81c095e700 1 -- 172.21.15.105:0/1465894421 >> 172.21.15.105:0/1465894421 conn(0x7f81bc0f2570 msgr2=0x7f81bc0f4990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:50.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.779+0000 7f81c095e700 1 -- 172.21.15.105:0/1465894421 shutdown_connections 2024-02-18T01:20:50.785 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.779+0000 7f81c095e700 1 -- 172.21.15.105:0/1465894421 wait complete. 2024-02-18T01:20:50.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 Processor -- start 2024-02-18T01:20:50.786 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- start start 2024-02-18T01:20:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81bc102c40 con 0x7f81bc1043b0 2024-02-18T01:20:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81bc102e10 con 0x7f81bc0f9630 2024-02-18T01:20:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f81bc103000 con 0x7f81bc0f8a80 2024-02-18T01:20:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81ba59c700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f81bc0f8a80 0x7f81bc1024a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:57018/0 (socket says 172.21.15.105:57018) 2024-02-18T01:20:50.787 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81ba59c700 1 -- 172.21.15.105:0/1562409167 learned_addr learned my addr 172.21.15.105:0/1562409167 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989594793 0 0) 0x7f81bc102c40 con 0x7f81bc1043b0 2024-02-18T01:20:50.788 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81a80033f0 con 0x7f81bc1043b0 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2982810764 0 0) 0x7f81bc103000 con 0x7f81bc0f8a80 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81bc102c40 con 0x7f81bc0f8a80 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999679933 0 0) 0x7f81bc102e10 con 0x7f81bc0f9630 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81bc103000 con 0x7f81bc0f9630 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3638735569 0 0) 0x7f81a80033f0 con 0x7f81bc1043b0 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81bc102e10 con 0x7f81bc1043b0 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f81b40021d0 con 0x7f81bc1043b0 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2463919636 0 0) 0x7f81bc102c40 con 0x7f81bc0f8a80 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f81a80033f0 con 0x7f81bc0f8a80 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f81ac001940 con 0x7f81bc0f8a80 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 956867478 0 0) 0x7f81bc102e10 con 0x7f81bc1043b0 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 >> v1:172.21.15.105:6790/0 conn(0x7f81bc0f8a80 legacy=0x7f81bc1024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:50.789 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 >> v1:172.21.15.155:6789/0 conn(0x7f81bc0f9630 legacy=0x7f81bc114e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81bc11cc80 con 0x7f81bc1043b0 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81bc119cb0 con 0x7f81bc1043b0 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f81b4003310 con 0x7f81bc1043b0 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f81b4003aa0 con 0x7f81bc1043b0 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.783+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f81bc11a120 con 0x7f81bc1043b0 2024-02-18T01:20:50.790 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.787+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81bc05f7b0 con 0x7f81bc1043b0 2024-02-18T01:20:50.791 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.787+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f81b4001bc0 con 0x7f81bc1043b0 2024-02-18T01:20:50.792 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.787+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f81b4076350 con 0x7f81bc1043b0 2024-02-18T01:20:50.800 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.795+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f81b4049f20 con 0x7f81bc1043b0 2024-02-18T01:20:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f81bc10a080 con 0x7f81bc1043b0 2024-02-18T01:20:50.982 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81b37fe700 1 -- 172.21.15.105:0/1562409167 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (unknown 832126871 0 2158320199) 0x7f81b4048390 con 0x7f81bc1043b0 2024-02-18T01:20:50.983 INFO:teuthology.orchestra.run.smithi105.stdout:55834574946 2024-02-18T01:20:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 >> v1:172.21.15.105:6800/2148176012 conn(0x7f81a8062220 legacy=0x7f81a80646e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 >> v1:172.21.15.105:6789/0 conn(0x7f81bc1043b0 legacy=0x7f81bc118530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 shutdown_connections 2024-02-18T01:20:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.979+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 >> 172.21.15.105:0/1562409167 conn(0x7f81bc0f2570 msgr2=0x7f81bc107840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:50.985 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.983+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 shutdown_connections 2024-02-18T01:20:50.986 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:50.983+0000 7f81c095e700 1 -- 172.21.15.105:0/1562409167 wait complete. 2024-02-18T01:20:51.660 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574933 got 55834574946 for osd.1 2024-02-18T01:20:51.660 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-02-18T01:20:51.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:51 smithi105 bash[19734]: cluster 2024-02-18T01:20:50.352629+0000 mgr.y (mgr.24475) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:51 smithi105 bash[19734]: audit 2024-02-18T01:20:50.982037+0000 mon.a (mon.0) 720 : audit [DBG] from='client.? v1:172.21.15.105:0/1562409167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T01:20:51.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:51 smithi105 bash[27669]: cluster 2024-02-18T01:20:50.352629+0000 mgr.y (mgr.24475) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:51.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:51 smithi105 bash[27669]: audit 2024-02-18T01:20:50.982037+0000 mon.a (mon.0) 720 : audit [DBG] from='client.? v1:172.21.15.105:0/1562409167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T01:20:51.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:51 smithi155 bash[20805]: cluster 2024-02-18T01:20:50.352629+0000 mgr.y (mgr.24475) 144 : cluster [DBG] pgmap v102: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:51.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:51 smithi155 bash[20805]: audit 2024-02-18T01:20:50.982037+0000 mon.a (mon.0) 720 : audit [DBG] from='client.? v1:172.21.15.105:0/1562409167' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-02-18T01:20:51.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:20:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:20:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:53 smithi105 bash[19734]: cluster 2024-02-18T01:20:52.354102+0000 mgr.y (mgr.24475) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:53 smithi105 bash[27669]: cluster 2024-02-18T01:20:52.354102+0000 mgr.y (mgr.24475) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:53 smithi155 bash[20805]: cluster 2024-02-18T01:20:52.354102+0000 mgr.y (mgr.24475) 145 : cluster [DBG] pgmap v103: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:55 smithi105 bash[27669]: cluster 2024-02-18T01:20:54.354798+0000 mgr.y (mgr.24475) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:55.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:55 smithi105 bash[19734]: cluster 2024-02-18T01:20:54.354798+0000 mgr.y (mgr.24475) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:55 smithi155 bash[20805]: cluster 2024-02-18T01:20:54.354798+0000 mgr.y (mgr.24475) 146 : cluster [DBG] pgmap v104: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:56.498 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:20:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:20:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:20:56] "GET /metrics HTTP/1.1" 200 172170 "" "Prometheus/2.33.4" 2024-02-18T01:20:57.742 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:20:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:20:57.779 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:57 smithi105 bash[27669]: cluster 2024-02-18T01:20:56.355496+0000 mgr.y (mgr.24475) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:57.780 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:57 smithi105 bash[19734]: cluster 2024-02-18T01:20:56.355496+0000 mgr.y (mgr.24475) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:57.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:57 smithi155 bash[20805]: cluster 2024-02-18T01:20:56.355496+0000 mgr.y (mgr.24475) 147 : cluster [DBG] pgmap v105: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:20:59.164 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:58 smithi105 bash[19734]: audit 2024-02-18T01:20:57.445788+0000 mgr.y (mgr.24475) 148 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:59.165 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:58 smithi105 bash[27669]: audit 2024-02-18T01:20:57.445788+0000 mgr.y (mgr.24475) 148 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:58 smithi155 bash[20805]: audit 2024-02-18T01:20:57.445788+0000 mgr.y (mgr.24475) 148 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:20:59.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- 172.21.15.105:0/918293945 >> v1:172.21.15.105:6789/0 conn(0x7f52100f9430 legacy=0x7f5210102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:59.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- 172.21.15.105:0/918293945 shutdown_connections 2024-02-18T01:20:59.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- 172.21.15.105:0/918293945 >> 172.21.15.105:0/918293945 conn(0x7f52100f2390 msgr2=0x7f52100f47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:59.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- 172.21.15.105:0/918293945 shutdown_connections 2024-02-18T01:20:59.396 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- 172.21.15.105:0/918293945 wait complete. 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 Processor -- start 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.391+0000 7f521572b700 1 -- start start 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5210117380 con 0x7f52100f8880 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5210118560 con 0x7f5210104210 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f5210119760 con 0x7f52100f9430 2024-02-18T01:20:59.397 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520ffff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f52100f8880 0x7f52100ff690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:35902/0 (socket says 172.21.15.105:35902) 2024-02-18T01:20:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520ffff700 1 -- 172.21.15.105:0/3586861359 learned_addr learned my addr 172.21.15.105:0/3586861359 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:20:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679247228 0 0) 0x7f5210119760 con 0x7f52100f9430 2024-02-18T01:20:59.398 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f8003630 con 0x7f52100f9430 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369415124 0 0) 0x7f5210117380 con 0x7f52100f8880 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5210119760 con 0x7f52100f8880 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3192496899 0 0) 0x7f5210118560 con 0x7f5210104210 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5210117380 con 0x7f5210104210 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3754585835 0 0) 0x7f51f8003630 con 0x7f52100f9430 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5210118560 con 0x7f52100f9430 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5200002c60 con 0x7f52100f9430 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1853979586 0 0) 0x7f5210119760 con 0x7f52100f8880 2024-02-18T01:20:59.399 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51f8003630 con 0x7f52100f8880 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2166666160 0 0) 0x7f5210117380 con 0x7f5210104210 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f5210119760 con 0x7f5210104210 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5204001880 con 0x7f52100f8880 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5208001b80 con 0x7f5210104210 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1664127512 0 0) 0x7f5210118560 con 0x7f52100f9430 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 >> v1:172.21.15.155:6789/0 conn(0x7f5210104210 legacy=0x7f5210115a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 >> v1:172.21.15.105:6789/0 conn(0x7f52100f8880 legacy=0x7f52100ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:59.400 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f521011a960 con 0x7f52100f9430 2024-02-18T01:20:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5210117570 con 0x7f52100f9430 2024-02-18T01:20:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f52101179e0 con 0x7f52100f9430 2024-02-18T01:20:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.395+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521005f5a0 con 0x7f52100f9430 2024-02-18T01:20:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.399+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f5200001230 con 0x7f52100f9430 2024-02-18T01:20:59.401 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.399+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f52000035e0 con 0x7f52100f9430 2024-02-18T01:20:59.403 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.399+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f5200016530 con 0x7f52100f9430 2024-02-18T01:20:59.405 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.403+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f5200001b30 con 0x7f52100f9430 2024-02-18T01:20:59.410 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.407+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f520004aa70 con 0x7f52100f9430 2024-02-18T01:20:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.591+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f52100613e0 con 0x7f52100f9430 2024-02-18T01:20:59.596 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.591+0000 7f520d7fa700 1 -- 172.21.15.105:0/3586861359 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (unknown 92182286 0 3577985045) 0x7f5200048e80 con 0x7f52100f9430 2024-02-18T01:20:59.597 INFO:teuthology.orchestra.run.smithi105.stdout:77309411420 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 >> v1:172.21.15.105:6800/2148176012 conn(0x7f51f8062560 legacy=0x7f51f8064a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 >> v1:172.21.15.105:6790/0 conn(0x7f52100f9430 legacy=0x7f52100ffda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 shutdown_connections 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 >> 172.21.15.105:0/3586861359 conn(0x7f52100f2390 msgr2=0x7f5210107680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 shutdown_connections 2024-02-18T01:20:59.599 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:20:59.595+0000 7f521572b700 1 -- 172.21.15.105:0/3586861359 wait complete. 2024-02-18T01:20:59.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:59 smithi105 bash[27669]: cluster 2024-02-18T01:20:58.357445+0000 mgr.y (mgr.24475) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:59.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:20:59 smithi105 bash[27669]: audit 2024-02-18T01:20:59.595823+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.105:0/3586861359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T01:20:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:59 smithi105 bash[19734]: cluster 2024-02-18T01:20:58.357445+0000 mgr.y (mgr.24475) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:20:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:20:59 smithi105 bash[19734]: audit 2024-02-18T01:20:59.595823+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.105:0/3586861359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T01:21:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:59 smithi155 bash[20805]: cluster 2024-02-18T01:20:58.357445+0000 mgr.y (mgr.24475) 149 : cluster [DBG] pgmap v106: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:20:59 smithi155 bash[20805]: audit 2024-02-18T01:20:59.595823+0000 mon.c (mon.2) 29 : audit [DBG] from='client.? v1:172.21.15.105:0/3586861359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-02-18T01:21:00.300 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411406 got 77309411420 for osd.2 2024-02-18T01:21:00.300 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-02-18T01:21:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:01 smithi105 bash[19734]: cluster 2024-02-18T01:21:00.358165+0000 mgr.y (mgr.24475) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:01 smithi105 bash[27669]: cluster 2024-02-18T01:21:00.358165+0000 mgr.y (mgr.24475) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:01 smithi155 bash[20805]: cluster 2024-02-18T01:21:00.358165+0000 mgr.y (mgr.24475) 150 : cluster [DBG] pgmap v107: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:04.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:03 smithi105 bash[19734]: cluster 2024-02-18T01:21:02.359554+0000 mgr.y (mgr.24475) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:03 smithi105 bash[27669]: cluster 2024-02-18T01:21:02.359554+0000 mgr.y (mgr.24475) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:03 smithi155 bash[20805]: cluster 2024-02-18T01:21:02.359554+0000 mgr.y (mgr.24475) 151 : cluster [DBG] pgmap v108: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:05.152 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:06.105 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:05 smithi105 bash[27669]: cluster 2024-02-18T01:21:04.360279+0000 mgr.y (mgr.24475) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:06.105 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:05 smithi105 bash[19734]: cluster 2024-02-18T01:21:04.360279+0000 mgr.y (mgr.24475) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:05 smithi155 bash[20805]: cluster 2024-02-18T01:21:04.360279+0000 mgr.y (mgr.24475) 152 : cluster [DBG] pgmap v109: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:06] "GET /metrics HTTP/1.1" 200 172167 "" "Prometheus/2.33.4" 2024-02-18T01:21:07.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:08.013 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.007+0000 7f30d60f6700 1 -- 172.21.15.105:0/3015036763 >> v1:172.21.15.105:6789/0 conn(0x7f30d006cc20 legacy=0x7f30d006b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.007+0000 7f30d60f6700 1 -- 172.21.15.105:0/3015036763 shutdown_connections 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.007+0000 7f30d60f6700 1 -- 172.21.15.105:0/3015036763 >> 172.21.15.105:0/3015036763 conn(0x7f30d00f3630 msgr2=0x7f30d00f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.007+0000 7f30d60f6700 1 -- 172.21.15.105:0/3015036763 shutdown_connections 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.007+0000 7f30d60f6700 1 -- 172.21.15.105:0/3015036763 wait complete. 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 Processor -- start 2024-02-18T01:21:08.014 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- start start 2024-02-18T01:21:08.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f30d011da10 con 0x7f30d010ace0 2024-02-18T01:21:08.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f30d011ebf0 con 0x7f30d006cc20 2024-02-18T01:21:08.016 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f30d011fdf0 con 0x7f30d006b880 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30cf7fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f30d006b880 0x7f30d0107040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:56000/0 (socket says 172.21.15.105:56000) 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30cf7fe700 1 -- 172.21.15.105:0/2631256735 learned_addr learned my addr 172.21.15.105:0/2631256735 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338214794 0 0) 0x7f30d011fdf0 con 0x7f30d006b880 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30bc003030 con 0x7f30d006b880 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1437077518 0 0) 0x7f30d011da10 con 0x7f30d010ace0 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d011fdf0 con 0x7f30d010ace0 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023718267 0 0) 0x7f30d011ebf0 con 0x7f30d006cc20 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30d011da10 con 0x7f30d006cc20 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3432932637 0 0) 0x7f30d011fdf0 con 0x7f30d010ace0 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30d011ebf0 con 0x7f30d010ace0 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f30c8001c40 con 0x7f30d010ace0 2024-02-18T01:21:08.017 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3116936551 0 0) 0x7f30bc003030 con 0x7f30d006b880 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f30d011fdf0 con 0x7f30d006b880 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f30c00015b0 con 0x7f30d006b880 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3720537726 0 0) 0x7f30d011ebf0 con 0x7f30d010ace0 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 >> v1:172.21.15.105:6790/0 conn(0x7f30d006b880 legacy=0x7f30d0107040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 >> v1:172.21.15.155:6789/0 conn(0x7f30d006cc20 legacy=0x7f30d0107750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d0120ff0 con 0x7f30d010ace0 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d011dc00 con 0x7f30d010ace0 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d011e070 con 0x7f30d010ace0 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f30c8002350 con 0x7f30d010ace0 2024-02-18T01:21:08.018 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f30c8003530 con 0x7f30d010ace0 2024-02-18T01:21:08.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.011+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b0004f20 con 0x7f30d010ace0 2024-02-18T01:21:08.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.015+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f30c80164a0 con 0x7f30d010ace0 2024-02-18T01:21:08.019 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.015+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f30c8075fe0 con 0x7f30d010ace0 2024-02-18T01:21:08.028 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.023+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f30c8049d00 con 0x7f30d010ace0 2024-02-18T01:21:08.207 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.203+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f30b00051d0 con 0x7f30d010ace0 2024-02-18T01:21:08.208 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.203+0000 7f30ccff9700 1 -- 172.21.15.105:0/2631256735 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (unknown 383520633 0 2832570704) 0x7f30c8048110 con 0x7f30d010ace0 2024-02-18T01:21:08.208 INFO:teuthology.orchestra.run.smithi105.stdout:107374182484 2024-02-18T01:21:08.211 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 >> v1:172.21.15.105:6800/2148176012 conn(0x7f30bc0620a0 legacy=0x7f30bc064560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 >> v1:172.21.15.105:6789/0 conn(0x7f30d010ace0 legacy=0x7f30d011c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 shutdown_connections 2024-02-18T01:21:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 >> 172.21.15.105:0/2631256735 conn(0x7f30d00f3630 msgr2=0x7f30d00f4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 shutdown_connections 2024-02-18T01:21:08.212 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:08.207+0000 7f30d60f6700 1 -- 172.21.15.105:0/2631256735 wait complete. 2024-02-18T01:21:08.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:07 smithi105 bash[27669]: cluster 2024-02-18T01:21:06.360850+0000 mgr.y (mgr.24475) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:08.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:07 smithi105 bash[27669]: audit 2024-02-18T01:21:07.542923+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:21:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:07 smithi105 bash[19734]: cluster 2024-02-18T01:21:06.360850+0000 mgr.y (mgr.24475) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:07 smithi105 bash[19734]: audit 2024-02-18T01:21:07.542923+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:21:08.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:07 smithi155 bash[20805]: cluster 2024-02-18T01:21:06.360850+0000 mgr.y (mgr.24475) 153 : cluster [DBG] pgmap v110: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:08.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:07 smithi155 bash[20805]: audit 2024-02-18T01:21:07.542923+0000 mon.a (mon.0) 721 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:21:08.908 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182471 got 107374182484 for osd.3 2024-02-18T01:21:08.909 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:07.456961+0000 mgr.y (mgr.24475) 154 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:07.854714+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:08.137063+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:08.138713+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:08.148194+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[19734]: audit 2024-02-18T01:21:08.207463+0000 mon.a (mon.0) 726 : audit [DBG] from='client.? v1:172.21.15.105:0/2631256735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:07.456961+0000 mgr.y (mgr.24475) 154 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:07.854714+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:08.137063+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:21:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:08.138713+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:21:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:08.148194+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:08 smithi105 bash[27669]: audit 2024-02-18T01:21:08.207463+0000 mon.a (mon.0) 726 : audit [DBG] from='client.? v1:172.21.15.105:0/2631256735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:07.456961+0000 mgr.y (mgr.24475) 154 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:07.854714+0000 mon.a (mon.0) 722 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:08.137063+0000 mon.a (mon.0) 723 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:08.138713+0000 mon.a (mon.0) 724 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:08.148194+0000 mon.a (mon.0) 725 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:21:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:08 smithi155 bash[20805]: audit 2024-02-18T01:21:08.207463+0000 mon.a (mon.0) 726 : audit [DBG] from='client.? v1:172.21.15.105:0/2631256735' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-02-18T01:21:10.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:09 smithi105 bash[27669]: cluster 2024-02-18T01:21:08.362386+0000 mgr.y (mgr.24475) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:10.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:09 smithi105 bash[19734]: cluster 2024-02-18T01:21:08.362386+0000 mgr.y (mgr.24475) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:10.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:09 smithi155 bash[20805]: cluster 2024-02-18T01:21:08.362386+0000 mgr.y (mgr.24475) 155 : cluster [DBG] pgmap v111: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:11 smithi105 bash[19734]: cluster 2024-02-18T01:21:10.363111+0000 mgr.y (mgr.24475) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:11 smithi105 bash[27669]: cluster 2024-02-18T01:21:10.363111+0000 mgr.y (mgr.24475) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:11 smithi155 bash[20805]: cluster 2024-02-18T01:21:10.363111+0000 mgr.y (mgr.24475) 156 : cluster [DBG] pgmap v112: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:13.741 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:13 smithi105 bash[19734]: cluster 2024-02-18T01:21:12.364518+0000 mgr.y (mgr.24475) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:13 smithi105 bash[27669]: cluster 2024-02-18T01:21:12.364518+0000 mgr.y (mgr.24475) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:13 smithi155 bash[20805]: cluster 2024-02-18T01:21:12.364518+0000 mgr.y (mgr.24475) 157 : cluster [DBG] pgmap v113: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:15 smithi105 bash[19734]: cluster 2024-02-18T01:21:14.365359+0000 mgr.y (mgr.24475) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:15 smithi105 bash[27669]: cluster 2024-02-18T01:21:14.365359+0000 mgr.y (mgr.24475) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:15 smithi155 bash[20805]: cluster 2024-02-18T01:21:14.365359+0000 mgr.y (mgr.24475) 158 : cluster [DBG] pgmap v114: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:16.831 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- 172.21.15.105:0/1889002128 >> v1:172.21.15.105:6789/0 conn(0x7f2094106640 legacy=0x7f2094108a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- 172.21.15.105:0/1889002128 shutdown_connections 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- 172.21.15.105:0/1889002128 >> 172.21.15.105:0/1889002128 conn(0x7f20940f3630 msgr2=0x7f20940f5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- 172.21.15.105:0/1889002128 shutdown_connections 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- 172.21.15.105:0/1889002128 wait complete. 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 Processor -- start 2024-02-18T01:21:16.832 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- start start 2024-02-18T01:21:16.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f2094119530 con 0x7f209406bc60 2024-02-18T01:21:16.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f209411a710 con 0x7f2094106640 2024-02-18T01:21:16.833 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.827+0000 7f209bbb3700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f209411b910 con 0x7f209406b170 2024-02-18T01:21:16.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f209994f700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f209406b170 0x7f20941017a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:51652/0 (socket says 172.21.15.105:51652) 2024-02-18T01:21:16.834 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f209994f700 1 -- 172.21.15.105:0/2240540032 learned_addr learned my addr 172.21.15.105:0/2240540032 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4157554570 0 0) 0x7f209411b910 con 0x7f209406b170 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f207c0033f0 con 0x7f209406b170 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652067513 0 0) 0x7f2094119530 con 0x7f209406bc60 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f209411b910 con 0x7f209406bc60 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849008204 0 0) 0x7f209411a710 con 0x7f2094106640 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2094119530 con 0x7f2094106640 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1103249511 0 0) 0x7f209411b910 con 0x7f209406bc60 2024-02-18T01:21:16.835 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f209411a710 con 0x7f209406bc60 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2378958997 0 0) 0x7f207c0033f0 con 0x7f209406b170 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f209411b910 con 0x7f209406b170 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2084001950 con 0x7f209406b170 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f20880018c0 con 0x7f209406bc60 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 670033984 0 0) 0x7f2094119530 con 0x7f2094106640 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f207c0033f0 con 0x7f2094106640 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f208c001360 con 0x7f2094106640 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3740763849 0 0) 0x7f209411a710 con 0x7f209406bc60 2024-02-18T01:21:16.836 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 >> v1:172.21.15.105:6790/0 conn(0x7f209406b170 legacy=0x7f20941017a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 >> v1:172.21.15.155:6789/0 conn(0x7f2094106640 legacy=0x7f2094117c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209411cb10 con 0x7f209406bc60 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209411bb00 con 0x7f209406bc60 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f209411bf70 con 0x7f209406bc60 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f2088002270 con 0x7f209406bc60 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f2088003280 con 0x7f209406bc60 2024-02-18T01:21:16.837 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.831+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2080004f20 con 0x7f209406bc60 2024-02-18T01:21:16.838 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.835+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f20880161d0 con 0x7f209406bc60 2024-02-18T01:21:16.839 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.835+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f2088075cd0 con 0x7f209406bc60 2024-02-18T01:21:16.845 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:16.843+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f20880498a0 con 0x7f209406bc60 2024-02-18T01:21:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:16] "GET /metrics HTTP/1.1" 200 172167 "" "Prometheus/2.33.4" 2024-02-18T01:21:17.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f20800051d0 con 0x7f209406bc60 2024-02-18T01:21:17.030 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f2092ffd700 1 -- 172.21.15.105:0/2240540032 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (unknown 1823589948 0 3462450949) 0x7f2088047cb0 con 0x7f209406bc60 2024-02-18T01:21:17.031 INFO:teuthology.orchestra.run.smithi105.stdout:133143986253 2024-02-18T01:21:17.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 >> v1:172.21.15.105:6800/2148176012 conn(0x7f207c062340 legacy=0x7f207c064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:17.032 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 >> v1:172.21.15.105:6789/0 conn(0x7f209406bc60 legacy=0x7f2094101eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:17.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 shutdown_connections 2024-02-18T01:21:17.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 >> 172.21.15.105:0/2240540032 conn(0x7f20940f3630 msgr2=0x7f2094109a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:17.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 shutdown_connections 2024-02-18T01:21:17.033 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:17.027+0000 7f209bbb3700 1 -- 172.21.15.105:0/2240540032 wait complete. 2024-02-18T01:21:17.679 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986240 got 133143986253 for osd.4 2024-02-18T01:21:17.679 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-02-18T01:21:17.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:17.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:17 smithi105 bash[19734]: cluster 2024-02-18T01:21:16.365946+0000 mgr.y (mgr.24475) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:17.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:17 smithi105 bash[19734]: audit 2024-02-18T01:21:17.029840+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? v1:172.21.15.105:0/2240540032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T01:21:17.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:17 smithi105 bash[27669]: cluster 2024-02-18T01:21:16.365946+0000 mgr.y (mgr.24475) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:17.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:17 smithi105 bash[27669]: audit 2024-02-18T01:21:17.029840+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? v1:172.21.15.105:0/2240540032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T01:21:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:17 smithi155 bash[20805]: cluster 2024-02-18T01:21:16.365946+0000 mgr.y (mgr.24475) 159 : cluster [DBG] pgmap v115: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:18.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:17 smithi155 bash[20805]: audit 2024-02-18T01:21:17.029840+0000 mon.a (mon.0) 727 : audit [DBG] from='client.? v1:172.21.15.105:0/2240540032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-02-18T01:21:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:18 smithi105 bash[19734]: audit 2024-02-18T01:21:17.468212+0000 mgr.y (mgr.24475) 160 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:18 smithi105 bash[27669]: audit 2024-02-18T01:21:17.468212+0000 mgr.y (mgr.24475) 160 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:18 smithi155 bash[20805]: audit 2024-02-18T01:21:17.468212+0000 mgr.y (mgr.24475) 160 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:19 smithi105 bash[19734]: cluster 2024-02-18T01:21:18.367413+0000 mgr.y (mgr.24475) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:19 smithi105 bash[27669]: cluster 2024-02-18T01:21:18.367413+0000 mgr.y (mgr.24475) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:19 smithi155 bash[20805]: cluster 2024-02-18T01:21:18.367413+0000 mgr.y (mgr.24475) 161 : cluster [DBG] pgmap v116: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:21 smithi105 bash[19734]: cluster 2024-02-18T01:21:20.368106+0000 mgr.y (mgr.24475) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:21 smithi105 bash[27669]: cluster 2024-02-18T01:21:20.368106+0000 mgr.y (mgr.24475) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:21 smithi155 bash[20805]: cluster 2024-02-18T01:21:20.368106+0000 mgr.y (mgr.24475) 162 : cluster [DBG] pgmap v117: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:22.522 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:23 smithi105 bash[19734]: cluster 2024-02-18T01:21:22.369667+0000 mgr.y (mgr.24475) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:24.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:23 smithi105 bash[27669]: cluster 2024-02-18T01:21:22.369667+0000 mgr.y (mgr.24475) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:23 smithi155 bash[20805]: cluster 2024-02-18T01:21:22.369667+0000 mgr.y (mgr.24475) 163 : cluster [DBG] pgmap v118: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:25.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 -- 172.21.15.105:0/2788870254 >> v1:172.21.15.105:6789/0 conn(0x7f78900fc7c0 legacy=0x7f78900fcba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:25.416 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 -- 172.21.15.105:0/2788870254 shutdown_connections 2024-02-18T01:21:25.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 -- 172.21.15.105:0/2788870254 >> 172.21.15.105:0/2788870254 conn(0x7f78900f23c0 msgr2=0x7f78900f47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:25.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 -- 172.21.15.105:0/2788870254 shutdown_connections 2024-02-18T01:21:25.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 -- 172.21.15.105:0/2788870254 wait complete. 2024-02-18T01:21:25.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.411+0000 7f789514d700 1 Processor -- start 2024-02-18T01:21:25.417 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- start start 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7890117370 con 0x7f7890103100 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7890118570 con 0x7f78900ff5c0 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7890119770 con 0x7f78900fc7c0 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788ffff700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f78900fc7c0 0x7f78900f9b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:48126/0 (socket says 172.21.15.105:48126) 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788ffff700 1 -- 172.21.15.105:0/2743970713 learned_addr learned my addr 172.21.15.105:0/2743970713 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:25.418 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789494c700 1 --1- 172.21.15.105:0/2743970713 >> v1:172.21.15.105:6789/0 conn(0x7f7890103100 0x7f7890115a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:46262/0 (socket says 172.21.15.105:46262) 2024-02-18T01:21:25.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614112090 0 0) 0x7f7890117370 con 0x7f7890103100 2024-02-18T01:21:25.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f787c003630 con 0x7f7890103100 2024-02-18T01:21:25.419 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2250974396 0 0) 0x7f7890119770 con 0x7f78900fc7c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7890117370 con 0x7f78900fc7c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199004438 0 0) 0x7f7890118570 con 0x7f78900ff5c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7890119770 con 0x7f78900ff5c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1361491090 0 0) 0x7f787c003630 con 0x7f7890103100 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7890118570 con 0x7f7890103100 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7888001bc0 con 0x7f7890103100 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1388504240 0 0) 0x7f7890117370 con 0x7f78900fc7c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f787c003630 con 0x7f78900fc7c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7878002d20 con 0x7f78900fc7c0 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 164607956 0 0) 0x7f7890118570 con 0x7f7890103100 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 >> v1:172.21.15.105:6790/0 conn(0x7f78900fc7c0 legacy=0x7f78900f9b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 >> v1:172.21.15.155:6789/0 conn(0x7f78900ff5c0 legacy=0x7f78900fa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:25.420 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789011a970 con 0x7f7890103100 2024-02-18T01:21:25.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7890117560 con 0x7f7890103100 2024-02-18T01:21:25.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f78901179d0 con 0x7f7890103100 2024-02-18T01:21:25.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f7888002690 con 0x7f7890103100 2024-02-18T01:21:25.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7888005d80 con 0x7f7890103100 2024-02-18T01:21:25.421 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.415+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7870004f20 con 0x7f7890103100 2024-02-18T01:21:25.422 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.419+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f7888018c90 con 0x7f7890103100 2024-02-18T01:21:25.423 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.419+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f78880788c0 con 0x7f7890103100 2024-02-18T01:21:25.429 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.423+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f788804c490 con 0x7f7890103100 2024-02-18T01:21:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f78700051d0 con 0x7f7890103100 2024-02-18T01:21:25.610 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f788d7fa700 1 -- 172.21.15.105:0/2743970713 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (unknown 2131975755 0 3762890464) 0x7f788804a8a0 con 0x7f7890103100 2024-02-18T01:21:25.610 INFO:teuthology.orchestra.run.smithi105.stdout:158913790024 2024-02-18T01:21:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 >> v1:172.21.15.105:6800/2148176012 conn(0x7f787c061ff0 legacy=0x7f787c0644b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 >> v1:172.21.15.105:6789/0 conn(0x7f7890103100 legacy=0x7f7890115a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 shutdown_connections 2024-02-18T01:21:25.612 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 >> 172.21.15.105:0/2743970713 conn(0x7f78900f23c0 msgr2=0x7f7890106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:25.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 shutdown_connections 2024-02-18T01:21:25.613 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:25.607+0000 7f789514d700 1 -- 172.21.15.105:0/2743970713 wait complete. 2024-02-18T01:21:25.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:25 smithi105 bash[19734]: cluster 2024-02-18T01:21:24.370406+0000 mgr.y (mgr.24475) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:25.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:25 smithi105 bash[19734]: audit 2024-02-18T01:21:25.609534+0000 mon.a (mon.0) 728 : audit [DBG] from='client.? v1:172.21.15.105:0/2743970713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T01:21:25.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:25 smithi105 bash[27669]: cluster 2024-02-18T01:21:24.370406+0000 mgr.y (mgr.24475) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:25.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:25 smithi105 bash[27669]: audit 2024-02-18T01:21:25.609534+0000 mon.a (mon.0) 728 : audit [DBG] from='client.? v1:172.21.15.105:0/2743970713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T01:21:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:25 smithi155 bash[20805]: cluster 2024-02-18T01:21:24.370406+0000 mgr.y (mgr.24475) 164 : cluster [DBG] pgmap v119: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:25 smithi155 bash[20805]: audit 2024-02-18T01:21:25.609534+0000 mon.a (mon.0) 728 : audit [DBG] from='client.? v1:172.21.15.105:0/2743970713' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-02-18T01:21:26.260 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790010 got 158913790024 for osd.5 2024-02-18T01:21:26.260 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-02-18T01:21:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:26] "GET /metrics HTTP/1.1" 200 172180 "" "Prometheus/2.33.4" 2024-02-18T01:21:27.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:28.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:27 smithi105 bash[19734]: cluster 2024-02-18T01:21:26.370917+0000 mgr.y (mgr.24475) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:27 smithi105 bash[27669]: cluster 2024-02-18T01:21:26.370917+0000 mgr.y (mgr.24475) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:27 smithi155 bash[20805]: cluster 2024-02-18T01:21:26.370917+0000 mgr.y (mgr.24475) 165 : cluster [DBG] pgmap v120: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:29.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:28 smithi105 bash[19734]: audit 2024-02-18T01:21:27.479554+0000 mgr.y (mgr.24475) 166 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:28 smithi105 bash[27669]: audit 2024-02-18T01:21:27.479554+0000 mgr.y (mgr.24475) 166 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:28 smithi155 bash[20805]: audit 2024-02-18T01:21:27.479554+0000 mgr.y (mgr.24475) 166 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:30.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:29 smithi105 bash[19734]: cluster 2024-02-18T01:21:28.372350+0000 mgr.y (mgr.24475) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:29 smithi105 bash[27669]: cluster 2024-02-18T01:21:28.372350+0000 mgr.y (mgr.24475) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:29 smithi155 bash[20805]: cluster 2024-02-18T01:21:28.372350+0000 mgr.y (mgr.24475) 167 : cluster [DBG] pgmap v121: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:31.105 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:32.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:31 smithi105 bash[19734]: cluster 2024-02-18T01:21:30.373051+0000 mgr.y (mgr.24475) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:31 smithi105 bash[27669]: cluster 2024-02-18T01:21:30.373051+0000 mgr.y (mgr.24475) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:31 smithi155 bash[20805]: cluster 2024-02-18T01:21:30.373051+0000 mgr.y (mgr.24475) 168 : cluster [DBG] pgmap v122: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 -- 172.21.15.105:0/3880193542 >> v1:172.21.15.105:6790/0 conn(0x7fc6841041f0 legacy=0x7fc6841066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 -- 172.21.15.105:0/3880193542 shutdown_connections 2024-02-18T01:21:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 -- 172.21.15.105:0/3880193542 >> 172.21.15.105:0/3880193542 conn(0x7fc6840f2390 msgr2=0x7fc6840f47b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 -- 172.21.15.105:0/3880193542 shutdown_connections 2024-02-18T01:21:33.925 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 -- 172.21.15.105:0/3880193542 wait complete. 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.919+0000 7fc68b59c700 1 Processor -- start 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- start start 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc684117030 con 0x7fc6840f9410 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc684118210 con 0x7fc6841041f0 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fc6841193f0 con 0x7fc6840f8920 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68a59a700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fc6840f8920 0x7fc6840ff210 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:42988/0 (socket says 172.21.15.105:42988) 2024-02-18T01:21:33.926 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68a59a700 1 -- 172.21.15.105:0/3038116694 learned_addr learned my addr 172.21.15.105:0/3038116694 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2712868662 0 0) 0x7fc6841193f0 con 0x7fc6840f8920 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6700033f0 con 0x7fc6840f8920 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2777784260 0 0) 0x7fc684118210 con 0x7fc6841041f0 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6841193f0 con 0x7fc6841041f0 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 699892588 0 0) 0x7fc6700033f0 con 0x7fc6840f8920 2024-02-18T01:21:33.927 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc684118210 con 0x7fc6840f8920 2024-02-18T01:21:33.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc6800019d0 con 0x7fc6840f8920 2024-02-18T01:21:33.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455458980 0 0) 0x7fc684117030 con 0x7fc6840f9410 2024-02-18T01:21:33.928 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6700033f0 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2724579743 0 0) 0x7fc6700033f0 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fc684117030 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc674001800 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 609586767 0 0) 0x7fc684117030 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 >> v1:172.21.15.105:6790/0 conn(0x7fc6840f8920 legacy=0x7fc6840ff210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 >> v1:172.21.15.155:6789/0 conn(0x7fc6841041f0 legacy=0x7fc684115880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68411a610 con 0x7fc6840f9410 2024-02-18T01:21:33.929 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc684118400 con 0x7fc6840f9410 2024-02-18T01:21:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fc684118870 con 0x7fc6840f9410 2024-02-18T01:21:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fc674001f50 con 0x7fc6840f9410 2024-02-18T01:21:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.923+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fc6740032a0 con 0x7fc6840f9410 2024-02-18T01:21:33.930 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.927+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc668004f20 con 0x7fc6840f9410 2024-02-18T01:21:33.932 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.927+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fc674001ab0 con 0x7fc6840f9410 2024-02-18T01:21:33.933 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.927+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7fc674075c60 con 0x7fc6840f9410 2024-02-18T01:21:33.940 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:33.935+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fc6740497d0 con 0x7fc6840f9410 2024-02-18T01:21:33.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:33 smithi105 bash[27669]: cluster 2024-02-18T01:21:32.374497+0000 mgr.y (mgr.24475) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:33.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:33 smithi105 bash[19734]: cluster 2024-02-18T01:21:32.374497+0000 mgr.y (mgr.24475) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.115+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fc6680051d0 con 0x7fc6840f9410 2024-02-18T01:21:34.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.115+0000 7fc67b7fe700 1 -- 172.21.15.105:0/3038116694 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (unknown 1274345170 0 1109052699) 0x7fc674047be0 con 0x7fc6840f9410 2024-02-18T01:21:34.120 INFO:teuthology.orchestra.run.smithi105.stdout:184683593794 2024-02-18T01:21:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 >> v1:172.21.15.105:6800/2148176012 conn(0x7fc670062240 legacy=0x7fc670064700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 >> v1:172.21.15.105:6789/0 conn(0x7fc6840f9410 legacy=0x7fc6840ff920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 shutdown_connections 2024-02-18T01:21:34.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 >> 172.21.15.105:0/3038116694 conn(0x7fc6840f2390 msgr2=0x7fc6840f4780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 shutdown_connections 2024-02-18T01:21:34.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:34.119+0000 7fc68b59c700 1 -- 172.21.15.105:0/3038116694 wait complete. 2024-02-18T01:21:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:33 smithi155 bash[20805]: cluster 2024-02-18T01:21:32.374497+0000 mgr.y (mgr.24475) 169 : cluster [DBG] pgmap v123: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:34.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593780 got 184683593794 for osd.6 2024-02-18T01:21:34.749 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-02-18T01:21:35.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:34 smithi105 bash[19734]: audit 2024-02-18T01:21:34.119556+0000 mon.a (mon.0) 729 : audit [DBG] from='client.? v1:172.21.15.105:0/3038116694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T01:21:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:34 smithi105 bash[27669]: audit 2024-02-18T01:21:34.119556+0000 mon.a (mon.0) 729 : audit [DBG] from='client.? v1:172.21.15.105:0/3038116694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T01:21:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:34 smithi155 bash[20805]: audit 2024-02-18T01:21:34.119556+0000 mon.a (mon.0) 729 : audit [DBG] from='client.? v1:172.21.15.105:0/3038116694' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-02-18T01:21:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:35 smithi105 bash[27669]: cluster 2024-02-18T01:21:34.375209+0000 mgr.y (mgr.24475) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:35 smithi105 bash[19734]: cluster 2024-02-18T01:21:34.375209+0000 mgr.y (mgr.24475) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:35 smithi155 bash[20805]: cluster 2024-02-18T01:21:34.375209+0000 mgr.y (mgr.24475) 170 : cluster [DBG] pgmap v124: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:36] "GET /metrics HTTP/1.1" 200 172179 "" "Prometheus/2.33.4" 2024-02-18T01:21:37.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:38.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:37 smithi105 bash[19734]: cluster 2024-02-18T01:21:36.375919+0000 mgr.y (mgr.24475) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:38.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:37 smithi105 bash[27669]: cluster 2024-02-18T01:21:36.375919+0000 mgr.y (mgr.24475) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:38.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:37 smithi155 bash[20805]: cluster 2024-02-18T01:21:36.375919+0000 mgr.y (mgr.24475) 171 : cluster [DBG] pgmap v125: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:38 smithi105 bash[19734]: audit 2024-02-18T01:21:37.486609+0000 mgr.y (mgr.24475) 172 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:38 smithi105 bash[27669]: audit 2024-02-18T01:21:37.486609+0000 mgr.y (mgr.24475) 172 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:38 smithi155 bash[20805]: audit 2024-02-18T01:21:37.486609+0000 mgr.y (mgr.24475) 172 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:39.581 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:39 smithi105 bash[19734]: cluster 2024-02-18T01:21:38.377488+0000 mgr.y (mgr.24475) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:40.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:39 smithi105 bash[27669]: cluster 2024-02-18T01:21:38.377488+0000 mgr.y (mgr.24475) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:40.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:39 smithi155 bash[20805]: cluster 2024-02-18T01:21:38.377488+0000 mgr.y (mgr.24475) 173 : cluster [DBG] pgmap v126: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:42.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:41 smithi105 bash[27669]: cluster 2024-02-18T01:21:40.378234+0000 mgr.y (mgr.24475) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:41 smithi105 bash[19734]: cluster 2024-02-18T01:21:40.378234+0000 mgr.y (mgr.24475) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:42.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:41 smithi155 bash[20805]: cluster 2024-02-18T01:21:40.378234+0000 mgr.y (mgr.24475) 174 : cluster [DBG] pgmap v127: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:42.476 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.470+0000 7f8e6008d700 1 -- 172.21.15.105:0/3011076121 >> v1:172.21.15.105:6789/0 conn(0x7f8e580fda10 legacy=0x7f8e580fddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.470+0000 7f8e6008d700 1 -- 172.21.15.105:0/3011076121 shutdown_connections 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.470+0000 7f8e6008d700 1 -- 172.21.15.105:0/3011076121 >> 172.21.15.105:0/3011076121 conn(0x7f8e580f3640 msgr2=0x7f8e580f5a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.470+0000 7f8e6008d700 1 -- 172.21.15.105:0/3011076121 shutdown_connections 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.470+0000 7f8e6008d700 1 -- 172.21.15.105:0/3011076121 wait complete. 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 Processor -- start 2024-02-18T01:21:42.477 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- start start 2024-02-18T01:21:42.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e581173f0 con 0x7f8e58100860 2024-02-18T01:21:42.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e581185d0 con 0x7f8e58104370 2024-02-18T01:21:42.478 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f8e581197d0 con 0x7f8e580fda10 2024-02-18T01:21:42.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e5de29700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f8e580fda10 0x7f8e580f9aa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:41674/0 (socket says 172.21.15.105:41674) 2024-02-18T01:21:42.479 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e5de29700 1 -- 172.21.15.105:0/2208351763 learned_addr learned my addr 172.21.15.105:0/2208351763 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3535466907 0 0) 0x7f8e581197d0 con 0x7f8e580fda10 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e40003630 con 0x7f8e580fda10 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221113514 0 0) 0x7f8e581173f0 con 0x7f8e58100860 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e581197d0 con 0x7f8e58100860 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 164465800 0 0) 0x7f8e40003630 con 0x7f8e580fda10 2024-02-18T01:21:42.480 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e581173f0 con 0x7f8e580fda10 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8e48002d20 con 0x7f8e580fda10 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897495927 0 0) 0x7f8e581185d0 con 0x7f8e58104370 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e40003630 con 0x7f8e58104370 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3221949830 0 0) 0x7f8e581197d0 con 0x7f8e58100860 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f8e581185d0 con 0x7f8e58100860 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8e4c001880 con 0x7f8e58100860 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2425459634 0 0) 0x7f8e581173f0 con 0x7f8e580fda10 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 >> v1:172.21.15.155:6789/0 conn(0x7f8e58104370 legacy=0x7f8e58115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 >> v1:172.21.15.105:6789/0 conn(0x7f8e58100860 legacy=0x7f8e580fa1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e5811a9d0 con 0x7f8e580fda10 2024-02-18T01:21:42.481 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e581175e0 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f8e48001b30 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f8e480038c0 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.474+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e58117ad0 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.478+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f8e48003b40 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.478+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e58060400 con 0x7f8e580fda10 2024-02-18T01:21:42.482 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.478+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f8e48078a00 con 0x7f8e580fda10 2024-02-18T01:21:42.487 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.482+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f8e4804c5d0 con 0x7f8e580fda10 2024-02-18T01:21:42.669 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.662+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f8e5805ecb0 con 0x7f8e580fda10 2024-02-18T01:21:42.670 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e56ffd700 1 -- 172.21.15.105:0/2208351763 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (unknown 1482059429 0 4094220714) 0x7f8e48052150 con 0x7f8e580fda10 2024-02-18T01:21:42.670 INFO:teuthology.orchestra.run.smithi105.stdout:210453397562 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 >> v1:172.21.15.105:6800/2148176012 conn(0x7f8e40062300 legacy=0x7f8e400647c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 >> v1:172.21.15.105:6790/0 conn(0x7f8e580fda10 legacy=0x7f8e580f9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 shutdown_connections 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 >> 172.21.15.105:0/2208351763 conn(0x7f8e580f3640 msgr2=0x7f8e581077d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 shutdown_connections 2024-02-18T01:21:42.672 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:42.666+0000 7f8e6008d700 1 -- 172.21.15.105:0/2208351763 wait complete. 2024-02-18T01:21:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:42 smithi105 bash[19734]: audit 2024-02-18T01:21:42.669353+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? v1:172.21.15.105:0/2208351763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T01:21:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:42 smithi105 bash[27669]: audit 2024-02-18T01:21:42.669353+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? v1:172.21.15.105:0/2208351763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T01:21:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:42 smithi155 bash[20805]: audit 2024-02-18T01:21:42.669353+0000 mon.c (mon.2) 30 : audit [DBG] from='client.? v1:172.21.15.105:0/2208351763' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-02-18T01:21:43.334 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397549 got 210453397562 for osd.7 2024-02-18T01:21:43.334 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-02-18T01:21:43.335 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T01:21:44.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:43 smithi105 bash[19734]: cluster 2024-02-18T01:21:42.379017+0000 mgr.y (mgr.24475) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:44.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:43 smithi105 bash[27669]: cluster 2024-02-18T01:21:42.379017+0000 mgr.y (mgr.24475) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:44.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:43 smithi155 bash[20805]: cluster 2024-02-18T01:21:42.379017+0000 mgr.y (mgr.24475) 175 : cluster [DBG] pgmap v128: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:44 smithi105 bash[19734]: cluster 2024-02-18T01:21:44.379676+0000 mgr.y (mgr.24475) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:44 smithi105 bash[27669]: cluster 2024-02-18T01:21:44.379676+0000 mgr.y (mgr.24475) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:44 smithi155 bash[20805]: cluster 2024-02-18T01:21:44.379676+0000 mgr.y (mgr.24475) 176 : cluster [DBG] pgmap v129: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:46] "GET /metrics HTTP/1.1" 200 172179 "" "Prometheus/2.33.4" 2024-02-18T01:21:47.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:47 smithi105 bash[19734]: cluster 2024-02-18T01:21:46.380402+0000 mgr.y (mgr.24475) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:47.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:47 smithi105 bash[27669]: cluster 2024-02-18T01:21:46.380402+0000 mgr.y (mgr.24475) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:47.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:47 smithi155 bash[20805]: cluster 2024-02-18T01:21:46.380402+0000 mgr.y (mgr.24475) 177 : cluster [DBG] pgmap v130: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:48.171 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:48.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:48 smithi105 bash[19734]: audit 2024-02-18T01:21:47.497989+0000 mgr.y (mgr.24475) 178 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:48.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:48 smithi105 bash[27669]: audit 2024-02-18T01:21:47.497989+0000 mgr.y (mgr.24475) 178 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:48.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:48 smithi155 bash[20805]: audit 2024-02-18T01:21:47.497989+0000 mgr.y (mgr.24475) 178 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:49.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:49 smithi105 bash[19734]: cluster 2024-02-18T01:21:48.381905+0000 mgr.y (mgr.24475) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:49.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:49 smithi105 bash[27669]: cluster 2024-02-18T01:21:48.381905+0000 mgr.y (mgr.24475) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:49.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:49 smithi155 bash[20805]: cluster 2024-02-18T01:21:48.381905+0000 mgr.y (mgr.24475) 179 : cluster [DBG] pgmap v131: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:51.117 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.110+0000 7f519464e700 1 -- 172.21.15.105:0/3629285929 >> v1:172.21.15.105:6789/0 conn(0x7f518c0fca90 legacy=0x7f518c10cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.110+0000 7f519464e700 1 -- 172.21.15.105:0/3629285929 shutdown_connections 2024-02-18T01:21:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.110+0000 7f519464e700 1 -- 172.21.15.105:0/3629285929 >> 172.21.15.105:0/3629285929 conn(0x7f518c0f2590 msgr2=0x7f518c0f49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.110+0000 7f519464e700 1 -- 172.21.15.105:0/3629285929 shutdown_connections 2024-02-18T01:21:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.110+0000 7f519464e700 1 -- 172.21.15.105:0/3629285929 wait complete. 2024-02-18T01:21:51.118 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 Processor -- start 2024-02-18T01:21:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- start start 2024-02-18T01:21:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f518c114fa0 con 0x7f518c0f7400 2024-02-18T01:21:51.119 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f518c11f070 con 0x7f518c0fca90 2024-02-18T01:21:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f518c120250 con 0x7f518c0f6850 2024-02-18T01:21:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f51923ea700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f518c0f6850 0x7f518c1143f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:40240/0 (socket says 172.21.15.105:40240) 2024-02-18T01:21:51.120 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f51923ea700 1 -- 172.21.15.105:0/876789531 learned_addr learned my addr 172.21.15.105:0/876789531 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:51.121 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1326920319 0 0) 0x7f518c120250 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51780033f0 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702782093 0 0) 0x7f518c114fa0 con 0x7f518c0f7400 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c120250 con 0x7f518c0f7400 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198616304 0 0) 0x7f518c11f070 con 0x7f518c0fca90 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f518c114fa0 con 0x7f518c0fca90 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2770401646 0 0) 0x7f51780033f0 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f518c11f070 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5174001980 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 382215348 0 0) 0x7f518c120250 con 0x7f518c0f7400 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51780033f0 con 0x7f518c0f7400 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f517c001d10 con 0x7f518c0f7400 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 224405366 0 0) 0x7f518c11f070 con 0x7f518c0f6850 2024-02-18T01:21:51.122 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 >> v1:172.21.15.155:6789/0 conn(0x7f518c0fca90 legacy=0x7f518c11c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 >> v1:172.21.15.105:6789/0 conn(0x7f518c0f7400 legacy=0x7f518c119210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c121450 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c11e0c0 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.114+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c11e530 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.118+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f5174001c60 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.118+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f5174005ae0 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.118+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c004f20 con 0x7f518c0f6850 2024-02-18T01:21:51.123 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.118+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f5174002270 con 0x7f518c0f6850 2024-02-18T01:21:51.124 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.118+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f51740785e0 con 0x7f518c0f6850 2024-02-18T01:21:51.132 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.126+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f517404c150 con 0x7f518c0f6850 2024-02-18T01:21:51.306 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.302+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 --> v1:172.21.15.105:6800/2148176012 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f516c000b90 con 0x7f51780620c0 2024-02-18T01:21:51.330 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.326+0000 7f518b7fe700 1 -- 172.21.15.105:0/876789531 <== mgr.24475 v1:172.21.15.105:6800/2148176012 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355196 (unknown 2965378022 0 13416449) 0x7f516c000b90 con 0x7f51780620c0 2024-02-18T01:21:51.340 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:21:51.344 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 >> v1:172.21.15.105:6800/2148176012 conn(0x7f51780620c0 legacy=0x7f5178064580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 >> v1:172.21.15.105:6790/0 conn(0x7f518c0f6850 legacy=0x7f518c1143f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 shutdown_connections 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 >> 172.21.15.105:0/876789531 conn(0x7f518c0f2590 msgr2=0x7f518c10ce60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 shutdown_connections 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:51.330+0000 7f519464e700 1 -- 172.21.15.105:0/876789531 wait complete. 2024-02-18T01:21:51.346 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-02-18T01:21:51.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:51 smithi105 bash[19734]: cluster 2024-02-18T01:21:50.382625+0000 mgr.y (mgr.24475) 180 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:51.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:51 smithi105 bash[27669]: cluster 2024-02-18T01:21:50.382625+0000 mgr.y (mgr.24475) 180 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:51.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:51 smithi155 bash[20805]: cluster 2024-02-18T01:21:50.382625+0000 mgr.y (mgr.24475) 180 : cluster [DBG] pgmap v132: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:51.998 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-02-18T01:21:50.382215+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1373,"num_read_kb":1093,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":660,"ondisk_log_size":660,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381824,"kb_used_data":5056,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579392,"statfs":{"total":767960285184,"available":765521297408,"internally_reserved":0,"allocated":5177344,"data_stored":1956979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005972"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544813+0000","last_change":"2024-02-18T01:17:16.835984+0000","last_active":"2024-02-18T01:18:36.544813+0000","last_peered":"2024-02-18T01:18:36.544813+0000","last_clean":"2024-02-18T01:18:36.544813+0000","last_became_active":"2024-02-18T01:17:16.835727+0000","last_became_peered":"2024-02-18T01:17:16.835727+0000","last_unstale":"2024-02-18T01:18:36.544813+0000","last_undegraded":"2024-02-18T01:18:36.544813+0000","last_fullsized":"2024-02-18T01:18:36.544813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543726+0000","last_change":"2024-02-18T01:17:14.832757+0000","last_active":"2024-02-18T01:18:36.543726+0000","last_peered":"2024-02-18T01:18:36.543726+0000","last_clean":"2024-02-18T01:18:36.543726+0000","last_became_active":"2024-02-18T01:17:14.832433+0000","last_became_peered":"2024-02-18T01:17:14.832433+0000","last_unstale":"2024-02-18T01:18:36.543726+0000","last_undegraded":"2024-02-18T01:18:36.543726+0000","last_fullsized":"2024-02-18T01:18:36.543726+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551995+0000","last_change":"2024-02-18T01:17:11.299424+0000","last_active":"2024-02-18T01:18:36.551995+0000","last_peered":"2024-02-18T01:18:36.551995+0000","last_clean":"2024-02-18T01:18:36.551995+0000","last_became_active":"2024-02-18T01:17:11.298491+0000","last_became_peered":"2024-02-18T01:17:11.298491+0000","last_unstale":"2024-02-18T01:18:36.551995+0000","last_undegraded":"2024-02-18T01:18:36.551995+0000","last_fullsized":"2024-02-18T01:18:36.551995+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'15","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.568556+0000","last_change":"2024-02-18T01:18:37.568556+0000","last_active":"2024-02-18T01:18:37.568556+0000","last_peered":"2024-02-18T01:18:37.568556+0000","last_clean":"2024-02-18T01:18:37.568556+0000","last_became_active":"2024-02-18T01:18:37.559301+0000","last_became_peered":"2024-02-18T01:18:37.559301+0000","last_unstale":"2024-02-18T01:18:37.568556+0000","last_undegraded":"2024-02-18T01:18:37.568556+0000","last_fullsized":"2024-02-18T01:18:37.568556+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586026+0000","last_change":"2024-02-18T01:17:17.095065+0000","last_active":"2024-02-18T01:18:36.586026+0000","last_peered":"2024-02-18T01:18:36.586026+0000","last_clean":"2024-02-18T01:18:36.586026+0000","last_became_active":"2024-02-18T01:17:17.094717+0000","last_became_peered":"2024-02-18T01:17:17.094717+0000","last_unstale":"2024-02-18T01:18:36.586026+0000","last_undegraded":"2024-02-18T01:18:36.586026+0000","last_fullsized":"2024-02-18T01:18:36.586026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544919+0000","last_change":"2024-02-18T01:17:14.834664+0000","last_active":"2024-02-18T01:18:36.544919+0000","last_peered":"2024-02-18T01:18:36.544919+0000","last_clean":"2024-02-18T01:18:36.544919+0000","last_became_active":"2024-02-18T01:17:14.834270+0000","last_became_peered":"2024-02-18T01:17:14.834270+0000","last_unstale":"2024-02-18T01:18:36.544919+0000","last_undegraded":"2024-02-18T01:18:36.544919+0000","last_fullsized":"2024-02-18T01:18:36.544919+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586963+0000","last_change":"2024-02-18T01:17:11.298980+0000","last_active":"2024-02-18T01:18:36.586963+0000","last_peered":"2024-02-18T01:18:36.586963+0000","last_clean":"2024-02-18T01:18:36.586963+0000","last_became_active":"2024-02-18T01:17:11.298577+0000","last_became_peered":"2024-02-18T01:17:11.298577+0000","last_unstale":"2024-02-18T01:18:36.586963+0000","last_undegraded":"2024-02-18T01:18:36.586963+0000","last_fullsized":"2024-02-18T01:18:36.586963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'39","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586084+0000","last_change":"2024-02-18T01:17:12.825782+0000","last_active":"2024-02-18T01:18:36.586084+0000","last_peered":"2024-02-18T01:18:36.586084+0000","last_clean":"2024-02-18T01:18:36.586084+0000","last_became_active":"2024-02-18T01:17:12.824964+0000","last_became_peered":"2024-02-18T01:17:12.824964+0000","last_unstale":"2024-02-18T01:18:36.586084+0000","last_undegraded":"2024-02-18T01:18:36.586084+0000","last_fullsized":"2024-02-18T01:18:36.586084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549076+0000","last_change":"2024-02-18T01:17:16.835292+0000","last_active":"2024-02-18T01:18:36.549076+0000","last_peered":"2024-02-18T01:18:36.549076+0000","last_clean":"2024-02-18T01:18:36.549076+0000","last_became_active":"2024-02-18T01:17:16.834962+0000","last_became_peered":"2024-02-18T01:17:16.834962+0000","last_unstale":"2024-02-18T01:18:36.549076+0000","last_undegraded":"2024-02-18T01:18:36.549076+0000","last_fullsized":"2024-02-18T01:18:36.549076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548967+0000","last_change":"2024-02-18T01:17:14.832828+0000","last_active":"2024-02-18T01:18:36.548967+0000","last_peered":"2024-02-18T01:18:36.548967+0000","last_clean":"2024-02-18T01:18:36.548967+0000","last_became_active":"2024-02-18T01:17:14.832503+0000","last_became_peered":"2024-02-18T01:17:14.832503+0000","last_unstale":"2024-02-18T01:18:36.548967+0000","last_undegraded":"2024-02-18T01:18:36.548967+0000","last_fullsized":"2024-02-18T01:18:36.548967+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549785+0000","last_change":"2024-02-18T01:17:10.705677+0000","last_active":"2024-02-18T01:18:36.549785+0000","last_peered":"2024-02-18T01:18:36.549785+0000","last_clean":"2024-02-18T01:18:36.549785+0000","last_became_active":"2024-02-18T01:17:10.705303+0000","last_became_peered":"2024-02-18T01:17:10.705303+0000","last_unstale":"2024-02-18T01:18:36.549785+0000","last_undegraded":"2024-02-18T01:18:36.549785+0000","last_fullsized":"2024-02-18T01:18:36.549785+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'21","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545409+0000","last_change":"2024-02-18T01:17:12.825755+0000","last_active":"2024-02-18T01:18:36.545409+0000","last_peered":"2024-02-18T01:18:36.545409+0000","last_clean":"2024-02-18T01:18:36.545409+0000","last_became_active":"2024-02-18T01:17:12.825377+0000","last_became_peered":"2024-02-18T01:17:12.825377+0000","last_unstale":"2024-02-18T01:18:36.545409+0000","last_undegraded":"2024-02-18T01:18:36.545409+0000","last_fullsized":"2024-02-18T01:18:36.545409+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548406+0000","last_change":"2024-02-18T01:17:16.839543+0000","last_active":"2024-02-18T01:18:36.548406+0000","last_peered":"2024-02-18T01:18:36.548406+0000","last_clean":"2024-02-18T01:18:36.548406+0000","last_became_active":"2024-02-18T01:17:16.839311+0000","last_became_peered":"2024-02-18T01:17:16.839311+0000","last_unstale":"2024-02-18T01:18:36.548406+0000","last_undegraded":"2024-02-18T01:18:36.548406+0000","last_fullsized":"2024-02-18T01:18:36.548406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494362+0000","last_change":"2024-02-18T01:17:14.827214+0000","last_active":"2024-02-18T01:18:37.494362+0000","last_peered":"2024-02-18T01:18:37.494362+0000","last_clean":"2024-02-18T01:18:37.494362+0000","last_became_active":"2024-02-18T01:17:14.826950+0000","last_became_peered":"2024-02-18T01:17:14.826950+0000","last_unstale":"2024-02-18T01:18:37.494362+0000","last_undegraded":"2024-02-18T01:18:37.494362+0000","last_fullsized":"2024-02-18T01:18:37.494362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549045+0000","last_change":"2024-02-18T01:17:10.706739+0000","last_active":"2024-02-18T01:18:36.549045+0000","last_peered":"2024-02-18T01:18:36.549045+0000","last_clean":"2024-02-18T01:18:36.549045+0000","last_became_active":"2024-02-18T01:17:10.706172+0000","last_became_peered":"2024-02-18T01:17:10.706172+0000","last_unstale":"2024-02-18T01:18:36.549045+0000","last_undegraded":"2024-02-18T01:18:36.549045+0000","last_fullsized":"2024-02-18T01:18:36.549045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'11","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545238+0000","last_change":"2024-02-18T01:17:12.826222+0000","last_active":"2024-02-18T01:18:36.545238+0000","last_peered":"2024-02-18T01:18:36.545238+0000","last_clean":"2024-02-18T01:18:36.545238+0000","last_became_active":"2024-02-18T01:17:12.825893+0000","last_became_peered":"2024-02-18T01:17:12.825893+0000","last_unstale":"2024-02-18T01:18:36.545238+0000","last_undegraded":"2024-02-18T01:18:36.545238+0000","last_fullsized":"2024-02-18T01:18:36.545238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546527+0000","last_change":"2024-02-18T01:17:17.095339+0000","last_active":"2024-02-18T01:18:36.546527+0000","last_peered":"2024-02-18T01:18:36.546527+0000","last_clean":"2024-02-18T01:18:36.546527+0000","last_became_active":"2024-02-18T01:17:17.094461+0000","last_became_peered":"2024-02-18T01:17:17.094461+0000","last_unstale":"2024-02-18T01:18:36.546527+0000","last_undegraded":"2024-02-18T01:18:36.546527+0000","last_fullsized":"2024-02-18T01:18:36.546527+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547559+0000","last_change":"2024-02-18T01:17:14.828698+0000","last_active":"2024-02-18T01:18:36.547559+0000","last_peered":"2024-02-18T01:18:36.547559+0000","last_clean":"2024-02-18T01:18:36.547559+0000","last_became_active":"2024-02-18T01:17:14.828373+0000","last_became_peered":"2024-02-18T01:17:14.828373+0000","last_unstale":"2024-02-18T01:18:36.547559+0000","last_undegraded":"2024-02-18T01:18:36.547559+0000","last_fullsized":"2024-02-18T01:18:36.547559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549938+0000","last_change":"2024-02-18T01:17:10.711503+0000","last_active":"2024-02-18T01:18:36.549938+0000","last_peered":"2024-02-18T01:18:36.549938+0000","last_clean":"2024-02-18T01:18:36.549938+0000","last_became_active":"2024-02-18T01:17:10.711207+0000","last_became_peered":"2024-02-18T01:17:10.711207+0000","last_unstale":"2024-02-18T01:18:36.549938+0000","last_undegraded":"2024-02-18T01:18:36.549938+0000","last_fullsized":"2024-02-18T01:18:36.549938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545594+0000","last_change":"2024-02-18T01:17:12.821480+0000","last_active":"2024-02-18T01:18:36.545594+0000","last_peered":"2024-02-18T01:18:36.545594+0000","last_clean":"2024-02-18T01:18:36.545594+0000","last_became_active":"2024-02-18T01:17:12.821084+0000","last_became_peered":"2024-02-18T01:17:12.821084+0000","last_unstale":"2024-02-18T01:18:36.545594+0000","last_undegraded":"2024-02-18T01:18:36.545594+0000","last_fullsized":"2024-02-18T01:18:36.545594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586399+0000","last_change":"2024-02-18T01:17:17.093600+0000","last_active":"2024-02-18T01:18:36.586399+0000","last_peered":"2024-02-18T01:18:36.586399+0000","last_clean":"2024-02-18T01:18:36.586399+0000","last_became_active":"2024-02-18T01:17:17.092661+0000","last_became_peered":"2024-02-18T01:17:17.092661+0000","last_unstale":"2024-02-18T01:18:36.586399+0000","last_undegraded":"2024-02-18T01:18:36.586399+0000","last_fullsized":"2024-02-18T01:18:36.586399+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545322+0000","last_change":"2024-02-18T01:17:14.823942+0000","last_active":"2024-02-18T01:18:36.545322+0000","last_peered":"2024-02-18T01:18:36.545322+0000","last_clean":"2024-02-18T01:18:36.545322+0000","last_became_active":"2024-02-18T01:17:14.823204+0000","last_became_peered":"2024-02-18T01:17:14.823204+0000","last_unstale":"2024-02-18T01:18:36.545322+0000","last_undegraded":"2024-02-18T01:18:36.545322+0000","last_fullsized":"2024-02-18T01:18:36.545322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545528+0000","last_change":"2024-02-18T01:17:11.301098+0000","last_active":"2024-02-18T01:18:36.545528+0000","last_peered":"2024-02-18T01:18:36.545528+0000","last_clean":"2024-02-18T01:18:36.545528+0000","last_became_active":"2024-02-18T01:17:11.300829+0000","last_became_peered":"2024-02-18T01:17:11.300829+0000","last_unstale":"2024-02-18T01:18:36.545528+0000","last_undegraded":"2024-02-18T01:18:36.545528+0000","last_fullsized":"2024-02-18T01:18:36.545528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'18","reported_seq":55,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586842+0000","last_change":"2024-02-18T01:17:12.830495+0000","last_active":"2024-02-18T01:18:36.586842+0000","last_peered":"2024-02-18T01:18:36.586842+0000","last_clean":"2024-02-18T01:18:36.586842+0000","last_became_active":"2024-02-18T01:17:12.830247+0000","last_became_peered":"2024-02-18T01:17:12.830247+0000","last_unstale":"2024-02-18T01:18:36.586842+0000","last_undegraded":"2024-02-18T01:18:36.586842+0000","last_fullsized":"2024-02-18T01:18:36.586842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493123+0000","last_change":"2024-02-18T01:17:16.839142+0000","last_active":"2024-02-18T01:18:37.493123+0000","last_peered":"2024-02-18T01:18:37.493123+0000","last_clean":"2024-02-18T01:18:37.493123+0000","last_became_active":"2024-02-18T01:17:16.838801+0000","last_became_peered":"2024-02-18T01:17:16.838801+0000","last_unstale":"2024-02-18T01:18:37.493123+0000","last_undegraded":"2024-02-18T01:18:37.493123+0000","last_fullsized":"2024-02-18T01:18:37.493123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134358+0000","last_change":"2024-02-18T01:17:14.838597+0000","last_active":"2024-02-18T01:21:39.134358+0000","last_peered":"2024-02-18T01:21:39.134358+0000","last_clean":"2024-02-18T01:21:39.134358+0000","last_became_active":"2024-02-18T01:17:14.836578+0000","last_became_peered":"2024-02-18T01:17:14.836578+0000","last_unstale":"2024-02-18T01:21:39.134358+0000","last_undegraded":"2024-02-18T01:21:39.134358+0000","last_fullsized":"2024-02-18T01:21:39.134358+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544050+0000","last_change":"2024-02-18T01:17:10.704953+0000","last_active":"2024-02-18T01:18:36.544050+0000","last_peered":"2024-02-18T01:18:36.544050+0000","last_clean":"2024-02-18T01:18:36.544050+0000","last_became_active":"2024-02-18T01:17:10.704586+0000","last_became_peered":"2024-02-18T01:17:10.704586+0000","last_unstale":"2024-02-18T01:18:36.544050+0000","last_undegraded":"2024-02-18T01:18:36.544050+0000","last_fullsized":"2024-02-18T01:18:36.544050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'16","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551572+0000","last_change":"2024-02-18T01:17:12.823321+0000","last_active":"2024-02-18T01:18:36.551572+0000","last_peered":"2024-02-18T01:18:36.551572+0000","last_clean":"2024-02-18T01:18:36.551572+0000","last_became_active":"2024-02-18T01:17:12.822982+0000","last_became_peered":"2024-02-18T01:17:12.822982+0000","last_unstale":"2024-02-18T01:18:36.551572+0000","last_undegraded":"2024-02-18T01:18:36.551572+0000","last_fullsized":"2024-02-18T01:18:36.551572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547651+0000","last_change":"2024-02-18T01:17:16.835943+0000","last_active":"2024-02-18T01:18:36.547651+0000","last_peered":"2024-02-18T01:18:36.547651+0000","last_clean":"2024-02-18T01:18:36.547651+0000","last_became_active":"2024-02-18T01:17:16.835626+0000","last_became_peered":"2024-02-18T01:17:16.835626+0000","last_unstale":"2024-02-18T01:18:36.547651+0000","last_undegraded":"2024-02-18T01:18:36.547651+0000","last_fullsized":"2024-02-18T01:18:36.547651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551822+0000","last_change":"2024-02-18T01:17:14.829269+0000","last_active":"2024-02-18T01:18:36.551822+0000","last_peered":"2024-02-18T01:18:36.551822+0000","last_clean":"2024-02-18T01:18:36.551822+0000","last_became_active":"2024-02-18T01:17:14.828815+0000","last_became_peered":"2024-02-18T01:17:14.828815+0000","last_unstale":"2024-02-18T01:18:36.551822+0000","last_undegraded":"2024-02-18T01:18:36.551822+0000","last_fullsized":"2024-02-18T01:18:36.551822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585385+0000","last_change":"2024-02-18T01:17:10.706768+0000","last_active":"2024-02-18T01:18:36.585385+0000","last_peered":"2024-02-18T01:18:36.585385+0000","last_clean":"2024-02-18T01:18:36.585385+0000","last_became_active":"2024-02-18T01:17:10.706273+0000","last_became_peered":"2024-02-18T01:17:10.706273+0000","last_unstale":"2024-02-18T01:18:36.585385+0000","last_undegraded":"2024-02-18T01:18:36.585385+0000","last_fullsized":"2024-02-18T01:18:36.585385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549489+0000","last_change":"2024-02-18T01:17:12.829234+0000","last_active":"2024-02-18T01:18:36.549489+0000","last_peered":"2024-02-18T01:18:36.549489+0000","last_clean":"2024-02-18T01:18:36.549489+0000","last_became_active":"2024-02-18T01:17:12.828666+0000","last_became_peered":"2024-02-18T01:17:12.828666+0000","last_unstale":"2024-02-18T01:18:36.549489+0000","last_undegraded":"2024-02-18T01:18:36.549489+0000","last_fullsized":"2024-02-18T01:18:36.549489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.492998+0000","last_change":"2024-02-18T01:17:16.838484+0000","last_active":"2024-02-18T01:18:37.492998+0000","last_peered":"2024-02-18T01:18:37.492998+0000","last_clean":"2024-02-18T01:18:37.492998+0000","last_became_active":"2024-02-18T01:17:16.838194+0000","last_became_peered":"2024-02-18T01:17:16.838194+0000","last_unstale":"2024-02-18T01:18:37.492998+0000","last_undegraded":"2024-02-18T01:18:37.492998+0000","last_fullsized":"2024-02-18T01:18:37.492998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547643+0000","last_change":"2024-02-18T01:17:14.836977+0000","last_active":"2024-02-18T01:18:36.547643+0000","last_peered":"2024-02-18T01:18:36.547643+0000","last_clean":"2024-02-18T01:18:36.547643+0000","last_became_active":"2024-02-18T01:17:14.836653+0000","last_became_peered":"2024-02-18T01:17:14.836653+0000","last_unstale":"2024-02-18T01:18:36.547643+0000","last_undegraded":"2024-02-18T01:18:36.547643+0000","last_fullsized":"2024-02-18T01:18:36.547643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.552014+0000","last_change":"2024-02-18T01:17:10.705385+0000","last_active":"2024-02-18T01:18:36.552014+0000","last_peered":"2024-02-18T01:18:36.552014+0000","last_clean":"2024-02-18T01:18:36.552014+0000","last_became_active":"2024-02-18T01:17:10.705059+0000","last_became_peered":"2024-02-18T01:17:10.705059+0000","last_unstale":"2024-02-18T01:18:36.552014+0000","last_undegraded":"2024-02-18T01:18:36.552014+0000","last_fullsized":"2024-02-18T01:18:36.552014+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585445+0000","last_change":"2024-02-18T01:17:12.828867+0000","last_active":"2024-02-18T01:18:36.585445+0000","last_peered":"2024-02-18T01:18:36.585445+0000","last_clean":"2024-02-18T01:18:36.585445+0000","last_became_active":"2024-02-18T01:17:12.828557+0000","last_became_peered":"2024-02-18T01:17:12.828557+0000","last_unstale":"2024-02-18T01:18:36.585445+0000","last_undegraded":"2024-02-18T01:18:36.585445+0000","last_fullsized":"2024-02-18T01:18:36.585445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585910+0000","last_change":"2024-02-18T01:17:17.093382+0000","last_active":"2024-02-18T01:18:36.585910+0000","last_peered":"2024-02-18T01:18:36.585910+0000","last_clean":"2024-02-18T01:18:36.585910+0000","last_became_active":"2024-02-18T01:17:17.092998+0000","last_became_peered":"2024-02-18T01:17:17.092998+0000","last_unstale":"2024-02-18T01:18:36.585910+0000","last_undegraded":"2024-02-18T01:18:36.585910+0000","last_fullsized":"2024-02-18T01:18:36.585910+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494868+0000","last_change":"2024-02-18T01:17:14.831650+0000","last_active":"2024-02-18T01:18:37.494868+0000","last_peered":"2024-02-18T01:18:37.494868+0000","last_clean":"2024-02-18T01:18:37.494868+0000","last_became_active":"2024-02-18T01:17:14.831382+0000","last_became_peered":"2024-02-18T01:17:14.831382+0000","last_unstale":"2024-02-18T01:18:37.494868+0000","last_undegraded":"2024-02-18T01:18:37.494868+0000","last_fullsized":"2024-02-18T01:18:37.494868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549574+0000","last_change":"2024-02-18T01:17:10.710694+0000","last_active":"2024-02-18T01:18:36.549574+0000","last_peered":"2024-02-18T01:18:36.549574+0000","last_clean":"2024-02-18T01:18:36.549574+0000","last_became_active":"2024-02-18T01:17:10.710111+0000","last_became_peered":"2024-02-18T01:17:10.710111+0000","last_unstale":"2024-02-18T01:18:36.549574+0000","last_undegraded":"2024-02-18T01:18:36.549574+0000","last_fullsized":"2024-02-18T01:18:36.549574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586025+0000","last_change":"2024-02-18T01:17:12.823207+0000","last_active":"2024-02-18T01:18:36.586025+0000","last_peered":"2024-02-18T01:18:36.586025+0000","last_clean":"2024-02-18T01:18:36.586025+0000","last_became_active":"2024-02-18T01:17:12.822888+0000","last_became_peered":"2024-02-18T01:17:12.822888+0000","last_unstale":"2024-02-18T01:18:36.586025+0000","last_undegraded":"2024-02-18T01:18:36.586025+0000","last_fullsized":"2024-02-18T01:18:36.586025+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548636+0000","last_change":"2024-02-18T01:17:16.834121+0000","last_active":"2024-02-18T01:18:36.548636+0000","last_peered":"2024-02-18T01:18:36.548636+0000","last_clean":"2024-02-18T01:18:36.548636+0000","last_became_active":"2024-02-18T01:17:16.833789+0000","last_became_peered":"2024-02-18T01:17:16.833789+0000","last_unstale":"2024-02-18T01:18:36.548636+0000","last_undegraded":"2024-02-18T01:18:36.548636+0000","last_fullsized":"2024-02-18T01:18:36.548636+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586173+0000","last_change":"2024-02-18T01:17:14.830922+0000","last_active":"2024-02-18T01:18:36.586173+0000","last_peered":"2024-02-18T01:18:36.586173+0000","last_clean":"2024-02-18T01:18:36.586173+0000","last_became_active":"2024-02-18T01:17:14.830580+0000","last_became_peered":"2024-02-18T01:17:14.830580+0000","last_unstale":"2024-02-18T01:18:36.586173+0000","last_undegraded":"2024-02-18T01:18:36.586173+0000","last_fullsized":"2024-02-18T01:18:36.586173+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494872+0000","last_change":"2024-02-18T01:17:10.716040+0000","last_active":"2024-02-18T01:18:37.494872+0000","last_peered":"2024-02-18T01:18:37.494872+0000","last_clean":"2024-02-18T01:18:37.494872+0000","last_became_active":"2024-02-18T01:17:10.715775+0000","last_became_peered":"2024-02-18T01:17:10.715775+0000","last_unstale":"2024-02-18T01:18:37.494872+0000","last_undegraded":"2024-02-18T01:18:37.494872+0000","last_fullsized":"2024-02-18T01:18:37.494872+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544692+0000","last_change":"2024-02-18T01:17:12.824681+0000","last_active":"2024-02-18T01:18:36.544692+0000","last_peered":"2024-02-18T01:18:36.544692+0000","last_clean":"2024-02-18T01:18:36.544692+0000","last_became_active":"2024-02-18T01:17:12.824381+0000","last_became_peered":"2024-02-18T01:17:12.824381+0000","last_unstale":"2024-02-18T01:18:36.544692+0000","last_undegraded":"2024-02-18T01:18:36.544692+0000","last_fullsized":"2024-02-18T01:18:36.544692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585683+0000","last_change":"2024-02-18T01:17:16.841341+0000","last_active":"2024-02-18T01:18:36.585683+0000","last_peered":"2024-02-18T01:18:36.585683+0000","last_clean":"2024-02-18T01:18:36.585683+0000","last_became_active":"2024-02-18T01:17:16.840941+0000","last_became_peered":"2024-02-18T01:17:16.840941+0000","last_unstale":"2024-02-18T01:18:36.585683+0000","last_undegraded":"2024-02-18T01:18:36.585683+0000","last_fullsized":"2024-02-18T01:18:36.585683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544049+0000","last_change":"2024-02-18T01:17:14.834100+0000","last_active":"2024-02-18T01:18:36.544049+0000","last_peered":"2024-02-18T01:18:36.544049+0000","last_clean":"2024-02-18T01:18:36.544049+0000","last_became_active":"2024-02-18T01:17:14.833849+0000","last_became_peered":"2024-02-18T01:17:14.833849+0000","last_unstale":"2024-02-18T01:18:36.544049+0000","last_undegraded":"2024-02-18T01:18:36.544049+0000","last_fullsized":"2024-02-18T01:18:36.544049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545601+0000","last_change":"2024-02-18T01:17:10.714192+0000","last_active":"2024-02-18T01:18:36.545601+0000","last_peered":"2024-02-18T01:18:36.545601+0000","last_clean":"2024-02-18T01:18:36.545601+0000","last_became_active":"2024-02-18T01:17:10.713903+0000","last_became_peered":"2024-02-18T01:17:10.713903+0000","last_unstale":"2024-02-18T01:18:36.545601+0000","last_undegraded":"2024-02-18T01:18:36.545601+0000","last_fullsized":"2024-02-18T01:18:36.545601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545580+0000","last_change":"2024-02-18T01:17:12.820899+0000","last_active":"2024-02-18T01:18:36.545580+0000","last_peered":"2024-02-18T01:18:36.545580+0000","last_clean":"2024-02-18T01:18:36.545580+0000","last_became_active":"2024-02-18T01:17:12.820556+0000","last_became_peered":"2024-02-18T01:17:12.820556+0000","last_unstale":"2024-02-18T01:18:36.545580+0000","last_undegraded":"2024-02-18T01:18:36.545580+0000","last_fullsized":"2024-02-18T01:18:36.545580+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551421+0000","last_change":"2024-02-18T01:17:16.840097+0000","last_active":"2024-02-18T01:18:36.551421+0000","last_peered":"2024-02-18T01:18:36.551421+0000","last_clean":"2024-02-18T01:18:36.551421+0000","last_became_active":"2024-02-18T01:17:16.839779+0000","last_became_peered":"2024-02-18T01:17:16.839779+0000","last_unstale":"2024-02-18T01:18:36.551421+0000","last_undegraded":"2024-02-18T01:18:36.551421+0000","last_fullsized":"2024-02-18T01:18:36.551421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":80,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134504+0000","last_change":"2024-02-18T01:17:14.838558+0000","last_active":"2024-02-18T01:21:39.134504+0000","last_peered":"2024-02-18T01:21:39.134504+0000","last_clean":"2024-02-18T01:21:39.134504+0000","last_became_active":"2024-02-18T01:17:14.838236+0000","last_became_peered":"2024-02-18T01:17:14.838236+0000","last_unstale":"2024-02-18T01:21:39.134504+0000","last_undegraded":"2024-02-18T01:21:39.134504+0000","last_fullsized":"2024-02-18T01:21:39.134504+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493811+0000","last_change":"2024-02-18T01:17:11.299796+0000","last_active":"2024-02-18T01:18:37.493811+0000","last_peered":"2024-02-18T01:18:37.493811+0000","last_clean":"2024-02-18T01:18:37.493811+0000","last_became_active":"2024-02-18T01:17:11.299257+0000","last_became_peered":"2024-02-18T01:17:11.299257+0000","last_unstale":"2024-02-18T01:18:37.493811+0000","last_undegraded":"2024-02-18T01:18:37.493811+0000","last_fullsized":"2024-02-18T01:18:37.493811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"61'22","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.587036+0000","last_change":"2024-02-18T01:17:12.826047+0000","last_active":"2024-02-18T01:18:36.587036+0000","last_peered":"2024-02-18T01:18:36.587036+0000","last_clean":"2024-02-18T01:18:36.587036+0000","last_became_active":"2024-02-18T01:17:12.825489+0000","last_became_peered":"2024-02-18T01:17:12.825489+0000","last_unstale":"2024-02-18T01:18:36.587036+0000","last_undegraded":"2024-02-18T01:18:36.587036+0000","last_fullsized":"2024-02-18T01:18:36.587036+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546346+0000","last_change":"2024-02-18T01:17:16.836056+0000","last_active":"2024-02-18T01:18:36.546346+0000","last_peered":"2024-02-18T01:18:36.546346+0000","last_clean":"2024-02-18T01:18:36.546346+0000","last_became_active":"2024-02-18T01:17:16.835730+0000","last_became_peered":"2024-02-18T01:17:16.835730+0000","last_unstale":"2024-02-18T01:18:36.546346+0000","last_undegraded":"2024-02-18T01:18:36.546346+0000","last_fullsized":"2024-02-18T01:18:36.546346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134266+0000","last_change":"2024-02-18T01:17:14.826870+0000","last_active":"2024-02-18T01:21:39.134266+0000","last_peered":"2024-02-18T01:21:39.134266+0000","last_clean":"2024-02-18T01:21:39.134266+0000","last_became_active":"2024-02-18T01:17:14.826541+0000","last_became_peered":"2024-02-18T01:17:14.826541+0000","last_unstale":"2024-02-18T01:21:39.134266+0000","last_undegraded":"2024-02-18T01:21:39.134266+0000","last_fullsized":"2024-02-18T01:21:39.134266+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548655+0000","last_change":"2024-02-18T01:17:10.710372+0000","last_active":"2024-02-18T01:18:36.548655+0000","last_peered":"2024-02-18T01:18:36.548655+0000","last_clean":"2024-02-18T01:18:36.548655+0000","last_became_active":"2024-02-18T01:17:10.709935+0000","last_became_peered":"2024-02-18T01:17:10.709935+0000","last_unstale":"2024-02-18T01:18:36.548655+0000","last_undegraded":"2024-02-18T01:18:36.548655+0000","last_fullsized":"2024-02-18T01:18:36.548655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"61'15","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549975+0000","last_change":"2024-02-18T01:17:12.818041+0000","last_active":"2024-02-18T01:18:36.549975+0000","last_peered":"2024-02-18T01:18:36.549975+0000","last_clean":"2024-02-18T01:18:36.549975+0000","last_became_active":"2024-02-18T01:17:12.817638+0000","last_became_peered":"2024-02-18T01:17:12.817638+0000","last_unstale":"2024-02-18T01:18:36.549975+0000","last_undegraded":"2024-02-18T01:18:36.549975+0000","last_fullsized":"2024-02-18T01:18:36.549975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545935+0000","last_change":"2024-02-18T01:17:16.838573+0000","last_active":"2024-02-18T01:18:36.545935+0000","last_peered":"2024-02-18T01:18:36.545935+0000","last_clean":"2024-02-18T01:18:36.545935+0000","last_became_active":"2024-02-18T01:17:16.837423+0000","last_became_peered":"2024-02-18T01:17:16.837423+0000","last_unstale":"2024-02-18T01:18:36.545935+0000","last_undegraded":"2024-02-18T01:18:36.545935+0000","last_fullsized":"2024-02-18T01:18:36.545935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586660+0000","last_change":"2024-02-18T01:17:14.827137+0000","last_active":"2024-02-18T01:18:36.586660+0000","last_peered":"2024-02-18T01:18:36.586660+0000","last_clean":"2024-02-18T01:18:36.586660+0000","last_became_active":"2024-02-18T01:17:14.826679+0000","last_became_peered":"2024-02-18T01:17:14.826679+0000","last_unstale":"2024-02-18T01:18:36.586660+0000","last_undegraded":"2024-02-18T01:18:36.586660+0000","last_fullsized":"2024-02-18T01:18:36.586660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494245+0000","last_change":"2024-02-18T01:17:10.715366+0000","last_active":"2024-02-18T01:18:37.494245+0000","last_peered":"2024-02-18T01:18:37.494245+0000","last_clean":"2024-02-18T01:18:37.494245+0000","last_became_active":"2024-02-18T01:17:10.715068+0000","last_became_peered":"2024-02-18T01:17:10.715068+0000","last_unstale":"2024-02-18T01:18:37.494245+0000","last_undegraded":"2024-02-18T01:18:37.494245+0000","last_fullsized":"2024-02-18T01:18:37.494245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'21","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551630+0000","last_change":"2024-02-18T01:17:12.824923+0000","last_active":"2024-02-18T01:18:36.551630+0000","last_peered":"2024-02-18T01:18:36.551630+0000","last_clean":"2024-02-18T01:18:36.551630+0000","last_became_active":"2024-02-18T01:17:12.824481+0000","last_became_peered":"2024-02-18T01:17:12.824481+0000","last_unstale":"2024-02-18T01:18:36.551630+0000","last_undegraded":"2024-02-18T01:18:36.551630+0000","last_fullsized":"2024-02-18T01:18:36.551630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494298+0000","last_change":"2024-02-18T01:17:17.092997+0000","last_active":"2024-02-18T01:18:37.494298+0000","last_peered":"2024-02-18T01:18:37.494298+0000","last_clean":"2024-02-18T01:18:37.494298+0000","last_became_active":"2024-02-18T01:17:17.092646+0000","last_became_peered":"2024-02-18T01:17:17.092646+0000","last_unstale":"2024-02-18T01:18:37.494298+0000","last_undegraded":"2024-02-18T01:18:37.494298+0000","last_fullsized":"2024-02-18T01:18:37.494298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549658+0000","last_change":"2024-02-18T01:17:14.838616+0000","last_active":"2024-02-18T01:18:36.549658+0000","last_peered":"2024-02-18T01:18:36.549658+0000","last_clean":"2024-02-18T01:18:36.549658+0000","last_became_active":"2024-02-18T01:17:14.838269+0000","last_became_peered":"2024-02-18T01:17:14.838269+0000","last_unstale":"2024-02-18T01:18:36.549658+0000","last_undegraded":"2024-02-18T01:18:36.549658+0000","last_fullsized":"2024-02-18T01:18:36.549658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547590+0000","last_change":"2024-02-18T01:17:10.708083+0000","last_active":"2024-02-18T01:18:36.547590+0000","last_peered":"2024-02-18T01:18:36.547590+0000","last_clean":"2024-02-18T01:18:36.547590+0000","last_became_active":"2024-02-18T01:17:10.707586+0000","last_became_peered":"2024-02-18T01:17:10.707586+0000","last_unstale":"2024-02-18T01:18:36.547590+0000","last_undegraded":"2024-02-18T01:18:36.547590+0000","last_fullsized":"2024-02-18T01:18:36.547590+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'12","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586556+0000","last_change":"2024-02-18T01:17:12.826487+0000","last_active":"2024-02-18T01:18:36.586556+0000","last_peered":"2024-02-18T01:18:36.586556+0000","last_clean":"2024-02-18T01:18:36.586556+0000","last_became_active":"2024-02-18T01:17:12.825723+0000","last_became_peered":"2024-02-18T01:17:12.825723+0000","last_unstale":"2024-02-18T01:18:36.586556+0000","last_undegraded":"2024-02-18T01:18:36.586556+0000","last_fullsized":"2024-02-18T01:18:36.586556+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549393+0000","last_change":"2024-02-18T01:17:17.093651+0000","last_active":"2024-02-18T01:18:36.549393+0000","last_peered":"2024-02-18T01:18:36.549393+0000","last_clean":"2024-02-18T01:18:36.549393+0000","last_became_active":"2024-02-18T01:17:17.092523+0000","last_became_peered":"2024-02-18T01:17:17.092523+0000","last_unstale":"2024-02-18T01:18:36.549393+0000","last_undegraded":"2024-02-18T01:18:36.549393+0000","last_fullsized":"2024-02-18T01:18:36.549393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494721+0000","last_change":"2024-02-18T01:17:10.716828+0000","last_active":"2024-02-18T01:18:37.494721+0000","last_peered":"2024-02-18T01:18:37.494721+0000","last_clean":"2024-02-18T01:18:37.494721+0000","last_became_active":"2024-02-18T01:17:10.716565+0000","last_became_peered":"2024-02-18T01:17:10.716565+0000","last_unstale":"2024-02-18T01:18:37.494721+0000","last_undegraded":"2024-02-18T01:18:37.494721+0000","last_fullsized":"2024-02-18T01:18:37.494721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"61'27","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548459+0000","last_change":"2024-02-18T01:17:12.819458+0000","last_active":"2024-02-18T01:18:36.548459+0000","last_peered":"2024-02-18T01:18:36.548459+0000","last_clean":"2024-02-18T01:18:36.548459+0000","last_became_active":"2024-02-18T01:17:12.819110+0000","last_became_peered":"2024-02-18T01:17:12.819110+0000","last_unstale":"2024-02-18T01:18:36.548459+0000","last_undegraded":"2024-02-18T01:18:36.548459+0000","last_fullsized":"2024-02-18T01:18:36.548459+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134523+0000","last_change":"2024-02-18T01:17:14.835396+0000","last_active":"2024-02-18T01:21:39.134523+0000","last_peered":"2024-02-18T01:21:39.134523+0000","last_clean":"2024-02-18T01:21:39.134523+0000","last_became_active":"2024-02-18T01:17:14.834731+0000","last_became_peered":"2024-02-18T01:17:14.834731+0000","last_unstale":"2024-02-18T01:21:39.134523+0000","last_undegraded":"2024-02-18T01:21:39.134523+0000","last_fullsized":"2024-02-18T01:21:39.134523+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585943+0000","last_change":"2024-02-18T01:17:16.842343+0000","last_active":"2024-02-18T01:18:36.585943+0000","last_peered":"2024-02-18T01:18:36.585943+0000","last_clean":"2024-02-18T01:18:36.585943+0000","last_became_active":"2024-02-18T01:17:16.841956+0000","last_became_peered":"2024-02-18T01:17:16.841956+0000","last_unstale":"2024-02-18T01:18:36.585943+0000","last_undegraded":"2024-02-18T01:18:36.585943+0000","last_fullsized":"2024-02-18T01:18:36.585943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494632+0000","last_change":"2024-02-18T01:17:10.717021+0000","last_active":"2024-02-18T01:18:37.494632+0000","last_peered":"2024-02-18T01:18:37.494632+0000","last_clean":"2024-02-18T01:18:37.494632+0000","last_became_active":"2024-02-18T01:17:10.716745+0000","last_became_peered":"2024-02-18T01:17:10.716745+0000","last_unstale":"2024-02-18T01:18:37.494632+0000","last_undegraded":"2024-02-18T01:18:37.494632+0000","last_fullsized":"2024-02-18T01:18:37.494632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'26","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545180+0000","last_change":"2024-02-18T01:17:12.829120+0000","last_active":"2024-02-18T01:18:36.545180+0000","last_peered":"2024-02-18T01:18:36.545180+0000","last_clean":"2024-02-18T01:18:36.545180+0000","last_became_active":"2024-02-18T01:17:12.828782+0000","last_became_peered":"2024-02-18T01:17:12.828782+0000","last_unstale":"2024-02-18T01:18:36.545180+0000","last_undegraded":"2024-02-18T01:18:36.545180+0000","last_fullsized":"2024-02-18T01:18:36.545180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544942+0000","last_change":"2024-02-18T01:17:14.821976+0000","last_active":"2024-02-18T01:18:36.544942+0000","last_peered":"2024-02-18T01:18:36.544942+0000","last_clean":"2024-02-18T01:18:36.544942+0000","last_became_active":"2024-02-18T01:17:14.821625+0000","last_became_peered":"2024-02-18T01:17:14.821625+0000","last_unstale":"2024-02-18T01:18:36.544942+0000","last_undegraded":"2024-02-18T01:18:36.544942+0000","last_fullsized":"2024-02-18T01:18:36.544942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.492999+0000","last_change":"2024-02-18T01:17:16.840263+0000","last_active":"2024-02-18T01:18:37.492999+0000","last_peered":"2024-02-18T01:18:37.492999+0000","last_clean":"2024-02-18T01:18:37.492999+0000","last_became_active":"2024-02-18T01:17:16.840012+0000","last_became_peered":"2024-02-18T01:17:16.840012+0000","last_unstale":"2024-02-18T01:18:37.492999+0000","last_undegraded":"2024-02-18T01:18:37.492999+0000","last_fullsized":"2024-02-18T01:18:37.492999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545548+0000","last_change":"2024-02-18T01:17:14.837962+0000","last_active":"2024-02-18T01:18:36.545548+0000","last_peered":"2024-02-18T01:18:36.545548+0000","last_clean":"2024-02-18T01:18:36.545548+0000","last_became_active":"2024-02-18T01:17:14.837597+0000","last_became_peered":"2024-02-18T01:17:14.837597+0000","last_unstale":"2024-02-18T01:18:36.545548+0000","last_undegraded":"2024-02-18T01:18:36.545548+0000","last_fullsized":"2024-02-18T01:18:36.545548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.495112+0000","last_change":"2024-02-18T01:17:10.713527+0000","last_active":"2024-02-18T01:18:37.495112+0000","last_peered":"2024-02-18T01:18:37.495112+0000","last_clean":"2024-02-18T01:18:37.495112+0000","last_became_active":"2024-02-18T01:17:10.713162+0000","last_became_peered":"2024-02-18T01:17:10.713162+0000","last_unstale":"2024-02-18T01:18:37.495112+0000","last_undegraded":"2024-02-18T01:18:37.495112+0000","last_fullsized":"2024-02-18T01:18:37.495112+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"61'25","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547975+0000","last_change":"2024-02-18T01:17:12.828783+0000","last_active":"2024-02-18T01:18:36.547975+0000","last_peered":"2024-02-18T01:18:36.547975+0000","last_clean":"2024-02-18T01:18:36.547975+0000","last_became_active":"2024-02-18T01:17:12.828435+0000","last_became_peered":"2024-02-18T01:17:12.828435+0000","last_unstale":"2024-02-18T01:18:36.547975+0000","last_undegraded":"2024-02-18T01:18:36.547975+0000","last_fullsized":"2024-02-18T01:18:36.547975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549338+0000","last_change":"2024-02-18T01:17:16.835057+0000","last_active":"2024-02-18T01:18:36.549338+0000","last_peered":"2024-02-18T01:18:36.549338+0000","last_clean":"2024-02-18T01:18:36.549338+0000","last_became_active":"2024-02-18T01:17:16.834338+0000","last_became_peered":"2024-02-18T01:17:16.834338+0000","last_unstale":"2024-02-18T01:18:36.549338+0000","last_undegraded":"2024-02-18T01:18:36.549338+0000","last_fullsized":"2024-02-18T01:18:36.549338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545095+0000","last_change":"2024-02-18T01:17:14.823641+0000","last_active":"2024-02-18T01:18:36.545095+0000","last_peered":"2024-02-18T01:18:36.545095+0000","last_clean":"2024-02-18T01:18:36.545095+0000","last_became_active":"2024-02-18T01:17:14.823276+0000","last_became_peered":"2024-02-18T01:17:14.823276+0000","last_unstale":"2024-02-18T01:18:36.545095+0000","last_undegraded":"2024-02-18T01:18:36.545095+0000","last_fullsized":"2024-02-18T01:18:36.545095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549160+0000","last_change":"2024-02-18T01:17:10.706508+0000","last_active":"2024-02-18T01:18:36.549160+0000","last_peered":"2024-02-18T01:18:36.549160+0000","last_clean":"2024-02-18T01:18:36.549160+0000","last_became_active":"2024-02-18T01:17:10.705707+0000","last_became_peered":"2024-02-18T01:17:10.705707+0000","last_unstale":"2024-02-18T01:18:36.549160+0000","last_undegraded":"2024-02-18T01:18:36.549160+0000","last_fullsized":"2024-02-18T01:18:36.549160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"61'15","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550033+0000","last_change":"2024-02-18T01:17:12.819522+0000","last_active":"2024-02-18T01:18:36.550033+0000","last_peered":"2024-02-18T01:18:36.550033+0000","last_clean":"2024-02-18T01:18:36.550033+0000","last_became_active":"2024-02-18T01:17:12.819201+0000","last_became_peered":"2024-02-18T01:17:12.819201+0000","last_unstale":"2024-02-18T01:18:36.550033+0000","last_undegraded":"2024-02-18T01:18:36.550033+0000","last_fullsized":"2024-02-18T01:18:36.550033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586671+0000","last_change":"2024-02-18T01:17:16.838837+0000","last_active":"2024-02-18T01:18:36.586671+0000","last_peered":"2024-02-18T01:18:36.586671+0000","last_clean":"2024-02-18T01:18:36.586671+0000","last_became_active":"2024-02-18T01:17:16.838546+0000","last_became_peered":"2024-02-18T01:17:16.838546+0000","last_unstale":"2024-02-18T01:18:36.586671+0000","last_undegraded":"2024-02-18T01:18:36.586671+0000","last_fullsized":"2024-02-18T01:18:36.586671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548888+0000","last_change":"2024-02-18T01:17:09.696709+0000","last_active":"2024-02-18T01:18:36.548888+0000","last_peered":"2024-02-18T01:18:36.548888+0000","last_clean":"2024-02-18T01:18:36.548888+0000","last_became_active":"2024-02-18T01:17:09.696408+0000","last_became_peered":"2024-02-18T01:17:09.696408+0000","last_unstale":"2024-02-18T01:18:36.548888+0000","last_undegraded":"2024-02-18T01:18:36.548888+0000","last_fullsized":"2024-02-18T01:18:36.548888+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545015+0000","last_change":"2024-02-18T01:17:10.703719+0000","last_active":"2024-02-18T01:18:36.545015+0000","last_peered":"2024-02-18T01:18:36.545015+0000","last_clean":"2024-02-18T01:18:36.545015+0000","last_became_active":"2024-02-18T01:17:10.703336+0000","last_became_peered":"2024-02-18T01:17:10.703336+0000","last_unstale":"2024-02-18T01:18:36.545015+0000","last_undegraded":"2024-02-18T01:18:36.545015+0000","last_fullsized":"2024-02-18T01:18:36.545015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"61'34","reported_seq":134,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134493+0000","last_change":"2024-02-18T01:17:12.822478+0000","last_active":"2024-02-18T01:21:39.134493+0000","last_peered":"2024-02-18T01:21:39.134493+0000","last_clean":"2024-02-18T01:21:39.134493+0000","last_became_active":"2024-02-18T01:17:12.822148+0000","last_became_peered":"2024-02-18T01:17:12.822148+0000","last_unstale":"2024-02-18T01:21:39.134493+0000","last_undegraded":"2024-02-18T01:21:39.134493+0000","last_fullsized":"2024-02-18T01:21:39.134493+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550355+0000","last_change":"2024-02-18T01:17:14.832997+0000","last_active":"2024-02-18T01:18:36.550355+0000","last_peered":"2024-02-18T01:18:36.550355+0000","last_clean":"2024-02-18T01:18:36.550355+0000","last_became_active":"2024-02-18T01:17:14.829833+0000","last_became_peered":"2024-02-18T01:17:14.829833+0000","last_unstale":"2024-02-18T01:18:36.550355+0000","last_undegraded":"2024-02-18T01:18:36.550355+0000","last_fullsized":"2024-02-18T01:18:36.550355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493228+0000","last_change":"2024-02-18T01:17:17.096876+0000","last_active":"2024-02-18T01:18:37.493228+0000","last_peered":"2024-02-18T01:18:37.493228+0000","last_clean":"2024-02-18T01:18:37.493228+0000","last_became_active":"2024-02-18T01:17:17.095788+0000","last_became_peered":"2024-02-18T01:17:17.095788+0000","last_unstale":"2024-02-18T01:18:37.493228+0000","last_undegraded":"2024-02-18T01:18:37.493228+0000","last_fullsized":"2024-02-18T01:18:37.493228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543788+0000","last_change":"2024-02-18T01:17:11.299544+0000","last_active":"2024-02-18T01:18:36.543788+0000","last_peered":"2024-02-18T01:18:36.543788+0000","last_clean":"2024-02-18T01:18:36.543788+0000","last_became_active":"2024-02-18T01:17:11.298038+0000","last_became_peered":"2024-02-18T01:17:11.298038+0000","last_unstale":"2024-02-18T01:18:36.543788+0000","last_undegraded":"2024-02-18T01:18:36.543788+0000","last_fullsized":"2024-02-18T01:18:36.543788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545286+0000","last_change":"2024-02-18T01:17:09.697042+0000","last_active":"2024-02-18T01:18:36.545286+0000","last_peered":"2024-02-18T01:18:36.545286+0000","last_clean":"2024-02-18T01:18:36.545286+0000","last_became_active":"2024-02-18T01:17:09.696626+0000","last_became_peered":"2024-02-18T01:17:09.696626+0000","last_unstale":"2024-02-18T01:18:36.545286+0000","last_undegraded":"2024-02-18T01:18:36.545286+0000","last_fullsized":"2024-02-18T01:18:36.545286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"61'19","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548034+0000","last_change":"2024-02-18T01:17:12.814141+0000","last_active":"2024-02-18T01:18:36.548034+0000","last_peered":"2024-02-18T01:18:36.548034+0000","last_clean":"2024-02-18T01:18:36.548034+0000","last_became_active":"2024-02-18T01:17:12.813750+0000","last_became_peered":"2024-02-18T01:17:12.813750+0000","last_unstale":"2024-02-18T01:18:36.548034+0000","last_undegraded":"2024-02-18T01:18:36.548034+0000","last_fullsized":"2024-02-18T01:18:36.548034+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546051+0000","last_change":"2024-02-18T01:17:14.827887+0000","last_active":"2024-02-18T01:18:36.546051+0000","last_peered":"2024-02-18T01:18:36.546051+0000","last_clean":"2024-02-18T01:18:36.546051+0000","last_became_active":"2024-02-18T01:17:14.827544+0000","last_became_peered":"2024-02-18T01:17:14.827544+0000","last_unstale":"2024-02-18T01:18:36.546051+0000","last_undegraded":"2024-02-18T01:18:36.546051+0000","last_fullsized":"2024-02-18T01:18:36.546051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"61'1","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550178+0000","last_change":"2024-02-18T01:17:16.839589+0000","last_active":"2024-02-18T01:18:36.550178+0000","last_peered":"2024-02-18T01:18:36.550178+0000","last_clean":"2024-02-18T01:18:36.550178+0000","last_became_active":"2024-02-18T01:17:16.839273+0000","last_became_peered":"2024-02-18T01:17:16.839273+0000","last_unstale":"2024-02-18T01:18:36.550178+0000","last_undegraded":"2024-02-18T01:18:36.550178+0000","last_fullsized":"2024-02-18T01:18:36.550178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494049+0000","last_change":"2024-02-18T01:16:57.389903+0000","last_active":"2024-02-18T01:18:37.494049+0000","last_peered":"2024-02-18T01:18:37.494049+0000","last_clean":"2024-02-18T01:18:37.494049+0000","last_became_active":"2024-02-18T01:16:57.388815+0000","last_became_peered":"2024-02-18T01:16:57.388815+0000","last_unstale":"2024-02-18T01:18:37.494049+0000","last_undegraded":"2024-02-18T01:18:37.494049+0000","last_fullsized":"2024-02-18T01:18:37.494049+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:13:29.577848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:13:29.577848+0000","last_clean_scrub_stamp":"2024-02-18T01:13:29.577848+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586129+0000","last_change":"2024-02-18T01:17:10.709874+0000","last_active":"2024-02-18T01:18:36.586129+0000","last_peered":"2024-02-18T01:18:36.586129+0000","last_clean":"2024-02-18T01:18:36.586129+0000","last_became_active":"2024-02-18T01:17:10.709527+0000","last_became_peered":"2024-02-18T01:17:10.709527+0000","last_unstale":"2024-02-18T01:18:36.586129+0000","last_undegraded":"2024-02-18T01:18:36.586129+0000","last_fullsized":"2024-02-18T01:18:36.586129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"61'14","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548924+0000","last_change":"2024-02-18T01:17:12.818782+0000","last_active":"2024-02-18T01:18:36.548924+0000","last_peered":"2024-02-18T01:18:36.548924+0000","last_clean":"2024-02-18T01:18:36.548924+0000","last_became_active":"2024-02-18T01:17:12.818420+0000","last_became_peered":"2024-02-18T01:17:12.818420+0000","last_unstale":"2024-02-18T01:18:36.548924+0000","last_undegraded":"2024-02-18T01:18:36.548924+0000","last_fullsized":"2024-02-18T01:18:36.548924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494932+0000","last_change":"2024-02-18T01:17:14.834512+0000","last_active":"2024-02-18T01:18:37.494932+0000","last_peered":"2024-02-18T01:18:37.494932+0000","last_clean":"2024-02-18T01:18:37.494932+0000","last_became_active":"2024-02-18T01:17:14.834201+0000","last_became_peered":"2024-02-18T01:17:14.834201+0000","last_unstale":"2024-02-18T01:18:37.494932+0000","last_undegraded":"2024-02-18T01:18:37.494932+0000","last_fullsized":"2024-02-18T01:18:37.494932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544070+0000","last_change":"2024-02-18T01:17:16.840858+0000","last_active":"2024-02-18T01:18:36.544070+0000","last_peered":"2024-02-18T01:18:36.544070+0000","last_clean":"2024-02-18T01:18:36.544070+0000","last_became_active":"2024-02-18T01:17:16.840569+0000","last_became_peered":"2024-02-18T01:17:16.840569+0000","last_unstale":"2024-02-18T01:18:36.544070+0000","last_undegraded":"2024-02-18T01:18:36.544070+0000","last_fullsized":"2024-02-18T01:18:36.544070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549613+0000","last_change":"2024-02-18T01:17:10.715267+0000","last_active":"2024-02-18T01:18:36.549613+0000","last_peered":"2024-02-18T01:18:36.549613+0000","last_clean":"2024-02-18T01:18:36.549613+0000","last_became_active":"2024-02-18T01:17:10.714928+0000","last_became_peered":"2024-02-18T01:17:10.714928+0000","last_unstale":"2024-02-18T01:18:36.549613+0000","last_undegraded":"2024-02-18T01:18:36.549613+0000","last_fullsized":"2024-02-18T01:18:36.549613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":292,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:45.510210+0000","last_change":"2024-02-18T01:17:09.701392+0000","last_active":"2024-02-18T01:21:45.510210+0000","last_peered":"2024-02-18T01:21:45.510210+0000","last_clean":"2024-02-18T01:21:45.510210+0000","last_became_active":"2024-02-18T01:17:09.701105+0000","last_became_peered":"2024-02-18T01:17:09.701105+0000","last_unstale":"2024-02-18T01:21:45.510210+0000","last_undegraded":"2024-02-18T01:21:45.510210+0000","last_fullsized":"2024-02-18T01:21:45.510210+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":269,"num_read_kb":264,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:38.602149+0000","last_change":"2024-02-18T01:18:38.602149+0000","last_active":"2024-02-18T01:18:38.602149+0000","last_peered":"2024-02-18T01:18:38.602149+0000","last_clean":"2024-02-18T01:18:38.602149+0000","last_became_active":"2024-02-18T01:18:37.562746+0000","last_became_peered":"2024-02-18T01:18:37.562746+0000","last_unstale":"2024-02-18T01:18:38.602149+0000","last_undegraded":"2024-02-18T01:18:38.602149+0000","last_fullsized":"2024-02-18T01:18:38.602149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550346+0000","last_change":"2024-02-18T01:17:14.834128+0000","last_active":"2024-02-18T01:18:36.550346+0000","last_peered":"2024-02-18T01:18:36.550346+0000","last_clean":"2024-02-18T01:18:36.550346+0000","last_became_active":"2024-02-18T01:17:14.833788+0000","last_became_peered":"2024-02-18T01:17:14.833788+0000","last_unstale":"2024-02-18T01:18:36.550346+0000","last_undegraded":"2024-02-18T01:18:36.550346+0000","last_fullsized":"2024-02-18T01:18:36.550346+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543623+0000","last_change":"2024-02-18T01:17:17.093645+0000","last_active":"2024-02-18T01:18:36.543623+0000","last_peered":"2024-02-18T01:18:36.543623+0000","last_clean":"2024-02-18T01:18:36.543623+0000","last_became_active":"2024-02-18T01:17:17.091841+0000","last_became_peered":"2024-02-18T01:17:17.091841+0000","last_unstale":"2024-02-18T01:18:36.543623+0000","last_undegraded":"2024-02-18T01:18:36.543623+0000","last_fullsized":"2024-02-18T01:18:36.543623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544240+0000","last_change":"2024-02-18T01:17:11.301831+0000","last_active":"2024-02-18T01:18:36.544240+0000","last_peered":"2024-02-18T01:18:36.544240+0000","last_clean":"2024-02-18T01:18:36.544240+0000","last_became_active":"2024-02-18T01:17:11.301510+0000","last_became_peered":"2024-02-18T01:17:11.301510+0000","last_unstale":"2024-02-18T01:18:36.544240+0000","last_undegraded":"2024-02-18T01:18:36.544240+0000","last_fullsized":"2024-02-18T01:18:36.544240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'25","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549889+0000","last_change":"2024-02-18T01:17:12.820612+0000","last_active":"2024-02-18T01:18:36.549889+0000","last_peered":"2024-02-18T01:18:36.549889+0000","last_clean":"2024-02-18T01:18:36.549889+0000","last_became_active":"2024-02-18T01:17:12.819570+0000","last_became_peered":"2024-02-18T01:17:12.819570+0000","last_unstale":"2024-02-18T01:18:36.549889+0000","last_undegraded":"2024-02-18T01:18:36.549889+0000","last_fullsized":"2024-02-18T01:18:36.549889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548166+0000","last_change":"2024-02-18T01:17:14.839830+0000","last_active":"2024-02-18T01:18:36.548166+0000","last_peered":"2024-02-18T01:18:36.548166+0000","last_clean":"2024-02-18T01:18:36.548166+0000","last_became_active":"2024-02-18T01:17:14.839502+0000","last_became_peered":"2024-02-18T01:17:14.839502+0000","last_unstale":"2024-02-18T01:18:36.548166+0000","last_undegraded":"2024-02-18T01:18:36.548166+0000","last_fullsized":"2024-02-18T01:18:36.548166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548725+0000","last_change":"2024-02-18T01:17:16.838705+0000","last_active":"2024-02-18T01:18:36.548725+0000","last_peered":"2024-02-18T01:18:36.548725+0000","last_clean":"2024-02-18T01:18:36.548725+0000","last_became_active":"2024-02-18T01:17:16.838355+0000","last_became_peered":"2024-02-18T01:17:16.838355+0000","last_unstale":"2024-02-18T01:18:36.548725+0000","last_undegraded":"2024-02-18T01:18:36.548725+0000","last_fullsized":"2024-02-18T01:18:36.548725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549894+0000","last_change":"2024-02-18T01:17:11.299454+0000","last_active":"2024-02-18T01:18:36.549894+0000","last_peered":"2024-02-18T01:18:36.549894+0000","last_clean":"2024-02-18T01:18:36.549894+0000","last_became_active":"2024-02-18T01:17:11.298786+0000","last_became_peered":"2024-02-18T01:17:11.298786+0000","last_unstale":"2024-02-18T01:18:36.549894+0000","last_undegraded":"2024-02-18T01:18:36.549894+0000","last_fullsized":"2024-02-18T01:18:36.549894+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"61'16","reported_seq":24,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:38.594750+0000","last_change":"2024-02-18T01:18:38.594750+0000","last_active":"2024-02-18T01:18:38.594750+0000","last_peered":"2024-02-18T01:18:38.594750+0000","last_clean":"2024-02-18T01:18:38.594750+0000","last_became_active":"2024-02-18T01:18:37.561512+0000","last_became_peered":"2024-02-18T01:18:37.561512+0000","last_unstale":"2024-02-18T01:18:38.594750+0000","last_undegraded":"2024-02-18T01:18:38.594750+0000","last_fullsized":"2024-02-18T01:18:38.594750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134482+0000","last_change":"2024-02-18T01:17:14.833420+0000","last_active":"2024-02-18T01:21:39.134482+0000","last_peered":"2024-02-18T01:21:39.134482+0000","last_clean":"2024-02-18T01:21:39.134482+0000","last_became_active":"2024-02-18T01:17:14.832798+0000","last_became_peered":"2024-02-18T01:17:14.832798+0000","last_unstale":"2024-02-18T01:21:39.134482+0000","last_undegraded":"2024-02-18T01:21:39.134482+0000","last_fullsized":"2024-02-18T01:21:39.134482+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493124+0000","last_change":"2024-02-18T01:17:17.096220+0000","last_active":"2024-02-18T01:18:37.493124+0000","last_peered":"2024-02-18T01:18:37.493124+0000","last_clean":"2024-02-18T01:18:37.493124+0000","last_became_active":"2024-02-18T01:17:17.095785+0000","last_became_peered":"2024-02-18T01:17:17.095785+0000","last_unstale":"2024-02-18T01:18:37.493124+0000","last_undegraded":"2024-02-18T01:18:37.493124+0000","last_fullsized":"2024-02-18T01:18:37.493124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551518+0000","last_change":"2024-02-18T01:17:10.704177+0000","last_active":"2024-02-18T01:18:36.551518+0000","last_peered":"2024-02-18T01:18:36.551518+0000","last_clean":"2024-02-18T01:18:36.551518+0000","last_became_active":"2024-02-18T01:17:10.703563+0000","last_became_peered":"2024-02-18T01:17:10.703563+0000","last_unstale":"2024-02-18T01:18:36.551518+0000","last_undegraded":"2024-02-18T01:18:36.551518+0000","last_fullsized":"2024-02-18T01:18:36.551518+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'20","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544958+0000","last_change":"2024-02-18T01:17:12.826509+0000","last_active":"2024-02-18T01:18:36.544958+0000","last_peered":"2024-02-18T01:18:36.544958+0000","last_clean":"2024-02-18T01:18:36.544958+0000","last_became_active":"2024-02-18T01:17:12.826187+0000","last_became_peered":"2024-02-18T01:17:12.826187+0000","last_unstale":"2024-02-18T01:18:36.544958+0000","last_undegraded":"2024-02-18T01:18:36.544958+0000","last_fullsized":"2024-02-18T01:18:36.544958+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134242+0000","last_change":"2024-02-18T01:17:14.824677+0000","last_active":"2024-02-18T01:21:39.134242+0000","last_peered":"2024-02-18T01:21:39.134242+0000","last_clean":"2024-02-18T01:21:39.134242+0000","last_became_active":"2024-02-18T01:17:14.824307+0000","last_became_peered":"2024-02-18T01:17:14.824307+0000","last_unstale":"2024-02-18T01:21:39.134242+0000","last_undegraded":"2024-02-18T01:21:39.134242+0000","last_fullsized":"2024-02-18T01:21:39.134242+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545702+0000","last_change":"2024-02-18T01:17:16.836790+0000","last_active":"2024-02-18T01:18:36.545702+0000","last_peered":"2024-02-18T01:18:36.545702+0000","last_clean":"2024-02-18T01:18:36.545702+0000","last_became_active":"2024-02-18T01:17:16.836495+0000","last_became_peered":"2024-02-18T01:17:16.836495+0000","last_unstale":"2024-02-18T01:18:36.545702+0000","last_undegraded":"2024-02-18T01:18:36.545702+0000","last_fullsized":"2024-02-18T01:18:36.545702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586501+0000","last_change":"2024-02-18T01:17:10.709186+0000","last_active":"2024-02-18T01:18:36.586501+0000","last_peered":"2024-02-18T01:18:36.586501+0000","last_clean":"2024-02-18T01:18:36.586501+0000","last_became_active":"2024-02-18T01:17:10.708852+0000","last_became_peered":"2024-02-18T01:17:10.708852+0000","last_unstale":"2024-02-18T01:18:36.586501+0000","last_undegraded":"2024-02-18T01:18:36.586501+0000","last_fullsized":"2024-02-18T01:18:36.586501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"61'30","reported_seq":79,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585520+0000","last_change":"2024-02-18T01:17:12.825039+0000","last_active":"2024-02-18T01:18:36.585520+0000","last_peered":"2024-02-18T01:18:36.585520+0000","last_clean":"2024-02-18T01:18:36.585520+0000","last_became_active":"2024-02-18T01:17:12.824666+0000","last_became_peered":"2024-02-18T01:17:12.824666+0000","last_unstale":"2024-02-18T01:18:36.585520+0000","last_undegraded":"2024-02-18T01:18:36.585520+0000","last_fullsized":"2024-02-18T01:18:36.585520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544713+0000","last_change":"2024-02-18T01:17:14.824194+0000","last_active":"2024-02-18T01:18:36.544713+0000","last_peered":"2024-02-18T01:18:36.544713+0000","last_clean":"2024-02-18T01:18:36.544713+0000","last_became_active":"2024-02-18T01:17:14.823349+0000","last_became_peered":"2024-02-18T01:17:14.823349+0000","last_unstale":"2024-02-18T01:18:36.544713+0000","last_undegraded":"2024-02-18T01:18:36.544713+0000","last_fullsized":"2024-02-18T01:18:36.544713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549693+0000","last_change":"2024-02-18T01:17:16.837880+0000","last_active":"2024-02-18T01:18:36.549693+0000","last_peered":"2024-02-18T01:18:36.549693+0000","last_clean":"2024-02-18T01:18:36.549693+0000","last_became_active":"2024-02-18T01:17:16.837631+0000","last_became_peered":"2024-02-18T01:17:16.837631+0000","last_unstale":"2024-02-18T01:18:36.549693+0000","last_undegraded":"2024-02-18T01:18:36.549693+0000","last_fullsized":"2024-02-18T01:18:36.549693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.135061+0000","last_change":"2024-02-18T01:17:14.832992+0000","last_active":"2024-02-18T01:21:39.135061+0000","last_peered":"2024-02-18T01:21:39.135061+0000","last_clean":"2024-02-18T01:21:39.135061+0000","last_became_active":"2024-02-18T01:17:14.832627+0000","last_became_peered":"2024-02-18T01:17:14.832627+0000","last_unstale":"2024-02-18T01:21:39.135061+0000","last_undegraded":"2024-02-18T01:21:39.135061+0000","last_fullsized":"2024-02-18T01:21:39.135061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585546+0000","last_change":"2024-02-18T01:17:10.710402+0000","last_active":"2024-02-18T01:18:36.585546+0000","last_peered":"2024-02-18T01:18:36.585546+0000","last_clean":"2024-02-18T01:18:36.585546+0000","last_became_active":"2024-02-18T01:17:10.710096+0000","last_became_peered":"2024-02-18T01:17:10.710096+0000","last_unstale":"2024-02-18T01:18:36.585546+0000","last_undegraded":"2024-02-18T01:18:36.585546+0000","last_fullsized":"2024-02-18T01:18:36.585546+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548084+0000","last_change":"2024-02-18T01:17:12.825237+0000","last_active":"2024-02-18T01:18:36.548084+0000","last_peered":"2024-02-18T01:18:36.548084+0000","last_clean":"2024-02-18T01:18:36.548084+0000","last_became_active":"2024-02-18T01:17:12.824964+0000","last_became_peered":"2024-02-18T01:17:12.824964+0000","last_unstale":"2024-02-18T01:18:36.548084+0000","last_undegraded":"2024-02-18T01:18:36.548084+0000","last_fullsized":"2024-02-18T01:18:36.548084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586435+0000","last_change":"2024-02-18T01:17:17.095044+0000","last_active":"2024-02-18T01:18:36.586435+0000","last_peered":"2024-02-18T01:18:36.586435+0000","last_clean":"2024-02-18T01:18:36.586435+0000","last_became_active":"2024-02-18T01:17:17.094630+0000","last_became_peered":"2024-02-18T01:17:17.094630+0000","last_unstale":"2024-02-18T01:18:36.586435+0000","last_undegraded":"2024-02-18T01:18:36.586435+0000","last_fullsized":"2024-02-18T01:18:36.586435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550286+0000","last_change":"2024-02-18T01:17:14.837188+0000","last_active":"2024-02-18T01:18:36.550286+0000","last_peered":"2024-02-18T01:18:36.550286+0000","last_clean":"2024-02-18T01:18:36.550286+0000","last_became_active":"2024-02-18T01:17:14.836859+0000","last_became_peered":"2024-02-18T01:17:14.836859+0000","last_unstale":"2024-02-18T01:18:36.550286+0000","last_undegraded":"2024-02-18T01:18:36.550286+0000","last_fullsized":"2024-02-18T01:18:36.550286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544750+0000","last_change":"2024-02-18T01:17:11.298329+0000","last_active":"2024-02-18T01:18:36.544750+0000","last_peered":"2024-02-18T01:18:36.544750+0000","last_clean":"2024-02-18T01:18:36.544750+0000","last_became_active":"2024-02-18T01:17:11.297607+0000","last_became_peered":"2024-02-18T01:17:11.297607+0000","last_unstale":"2024-02-18T01:18:36.544750+0000","last_undegraded":"2024-02-18T01:18:36.544750+0000","last_fullsized":"2024-02-18T01:18:36.544750+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546296+0000","last_change":"2024-02-18T01:17:12.826584+0000","last_active":"2024-02-18T01:18:36.546296+0000","last_peered":"2024-02-18T01:18:36.546296+0000","last_clean":"2024-02-18T01:18:36.546296+0000","last_became_active":"2024-02-18T01:17:12.826226+0000","last_became_peered":"2024-02-18T01:17:12.826226+0000","last_unstale":"2024-02-18T01:18:36.546296+0000","last_undegraded":"2024-02-18T01:18:36.546296+0000","last_fullsized":"2024-02-18T01:18:36.546296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545858+0000","last_change":"2024-02-18T01:17:16.836838+0000","last_active":"2024-02-18T01:18:36.545858+0000","last_peered":"2024-02-18T01:18:36.545858+0000","last_clean":"2024-02-18T01:18:36.545858+0000","last_became_active":"2024-02-18T01:17:16.836509+0000","last_became_peered":"2024-02-18T01:17:16.836509+0000","last_unstale":"2024-02-18T01:18:36.545858+0000","last_undegraded":"2024-02-18T01:18:36.545858+0000","last_fullsized":"2024-02-18T01:18:36.545858+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544527+0000","last_change":"2024-02-18T01:17:14.828948+0000","last_active":"2024-02-18T01:18:36.544527+0000","last_peered":"2024-02-18T01:18:36.544527+0000","last_clean":"2024-02-18T01:18:36.544527+0000","last_became_active":"2024-02-18T01:17:14.828368+0000","last_became_peered":"2024-02-18T01:17:14.828368+0000","last_unstale":"2024-02-18T01:18:36.544527+0000","last_undegraded":"2024-02-18T01:18:36.544527+0000","last_fullsized":"2024-02-18T01:18:36.544527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"54'1","reported_seq":25,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548265+0000","last_change":"2024-02-18T01:17:10.707232+0000","last_active":"2024-02-18T01:18:36.548265+0000","last_peered":"2024-02-18T01:18:36.548265+0000","last_clean":"2024-02-18T01:18:36.548265+0000","last_became_active":"2024-02-18T01:17:10.706178+0000","last_became_peered":"2024-02-18T01:17:10.706178+0000","last_unstale":"2024-02-18T01:18:36.548265+0000","last_undegraded":"2024-02-18T01:18:36.548265+0000","last_fullsized":"2024-02-18T01:18:36.548265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546247+0000","last_change":"2024-02-18T01:17:12.822070+0000","last_active":"2024-02-18T01:18:36.546247+0000","last_peered":"2024-02-18T01:18:36.546247+0000","last_clean":"2024-02-18T01:18:36.546247+0000","last_became_active":"2024-02-18T01:17:12.821732+0000","last_became_peered":"2024-02-18T01:17:12.821732+0000","last_unstale":"2024-02-18T01:18:36.546247+0000","last_undegraded":"2024-02-18T01:18:36.546247+0000","last_fullsized":"2024-02-18T01:18:36.546247+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547064+0000","last_change":"2024-02-18T01:17:16.837596+0000","last_active":"2024-02-18T01:18:36.547064+0000","last_peered":"2024-02-18T01:18:36.547064+0000","last_clean":"2024-02-18T01:18:36.547064+0000","last_became_active":"2024-02-18T01:17:16.836366+0000","last_became_peered":"2024-02-18T01:17:16.836366+0000","last_unstale":"2024-02-18T01:18:36.547064+0000","last_undegraded":"2024-02-18T01:18:36.547064+0000","last_fullsized":"2024-02-18T01:18:36.547064+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":77,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:39.134386+0000","last_change":"2024-02-18T01:17:14.829883+0000","last_active":"2024-02-18T01:21:39.134386+0000","last_peered":"2024-02-18T01:21:39.134386+0000","last_clean":"2024-02-18T01:21:39.134386+0000","last_became_active":"2024-02-18T01:17:14.828822+0000","last_became_peered":"2024-02-18T01:17:14.828822+0000","last_unstale":"2024-02-18T01:21:39.134386+0000","last_undegraded":"2024-02-18T01:21:39.134386+0000","last_fullsized":"2024-02-18T01:21:39.134386+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586170+0000","last_change":"2024-02-18T01:17:10.704860+0000","last_active":"2024-02-18T01:18:36.586170+0000","last_peered":"2024-02-18T01:18:36.586170+0000","last_clean":"2024-02-18T01:18:36.586170+0000","last_became_active":"2024-02-18T01:17:10.704554+0000","last_became_peered":"2024-02-18T01:17:10.704554+0000","last_unstale":"2024-02-18T01:18:36.586170+0000","last_undegraded":"2024-02-18T01:18:36.586170+0000","last_fullsized":"2024-02-18T01:18:36.586170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545605+0000","last_change":"2024-02-18T01:17:12.825708+0000","last_active":"2024-02-18T01:18:36.545605+0000","last_peered":"2024-02-18T01:18:36.545605+0000","last_clean":"2024-02-18T01:18:36.545605+0000","last_became_active":"2024-02-18T01:17:12.825434+0000","last_became_peered":"2024-02-18T01:17:12.825434+0000","last_unstale":"2024-02-18T01:18:36.545605+0000","last_undegraded":"2024-02-18T01:18:36.545605+0000","last_fullsized":"2024-02-18T01:18:36.545605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":585,"ondisk_log_size":585,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":269,"num_read_kb":264,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397564,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":244356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86699999999999999,"15min":0.80500000000000005},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.878,"15min":0.84199999999999997},"min":{"1min":0.66600000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":1.0780000000000001}]},{"osd":1,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0720000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91800000000000004,"15min":0.89900000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3740000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.76200000000000001}]},{"osd":2,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.872,"15min":0.80700000000000005},"min":{"1min":0.63,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.266,"15min":1.266},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91600000000000004,"15min":0.86199999999999999},"min":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2929999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.875}]},{"osd":3,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.92900000000000005,"15min":0.88900000000000001},"min":{"1min":0.70999999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.01,"15min":0.995},"min":{"1min":0.80700000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2310000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.91000000000000003}]},{"osd":4,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.84599999999999997,"15min":0.81499999999999995},"min":{"1min":0.65700000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.194,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.046,"5min":0.99199999999999999,"15min":0.94999999999999996},"min":{"1min":0.70799999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2809999999999999,"5min":1.456,"15min":1.456},"last":0.97499999999999998}]},{"osd":5,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94299999999999995,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.331,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.0089999999999999},{"interface":"front","average":{"1min":1,"5min":0.95199999999999996,"15min":0.92200000000000004},"min":{"1min":0.65200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.226,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":1.3720000000000001}]},{"osd":6,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.048,"15min":1.04},"min":{"1min":0.76600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.335,"5min":1.508,"15min":1.508},"last":1.119},{"interface":"front","average":{"1min":1.073,"5min":1.0129999999999999,"15min":0.98699999999999999},"min":{"1min":0.81200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.484,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.288}]}]},{"osd":6,"up_from":43,"seq":184683593797,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297708,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447444,"statfs":{"total":95995035648,"available":95690182656,"internally_reserved":0,"allocated":626688,"data_stored":243959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.65200000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.29,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.89800000000000002,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3580000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.76800000000000002}]},{"osd":1,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89300000000000002,"15min":0.873},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.113,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.121},{"interface":"front","average":{"1min":0.879,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.0700000000000001}]},{"osd":2,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.97399999999999998,"15min":0.96199999999999997},"min":{"1min":0.85999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.995,"5min":0.997,"15min":0.98599999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2949999999999999,"5min":1.528,"15min":1.528},"last":0.97799999999999998}]},{"osd":3,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.71099999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.26,"5min":1.542,"15min":1.542},"last":1.2450000000000001},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95399999999999996,"15min":0.98099999999999998},"min":{"1min":0.75,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.95199999999999996}]},{"osd":4,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.93600000000000005},"min":{"1min":0.65000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3400000000000001,"5min":1.738,"15min":1.738},"last":0.86299999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.0269999999999999},"min":{"1min":0.69099999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.3340000000000001}]},{"osd":5,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.008},"min":{"1min":0.68799999999999994,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.109,"15min":1.0820000000000001},"min":{"1min":0.90900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.4059999999999999}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1220000000000001,"15min":1.111},"min":{"1min":0.80500000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4339999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.278},{"interface":"front","average":{"1min":1.238,"5min":1.204,"15min":1.173},"min":{"1min":1.0449999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.464,"5min":1.742,"15min":1.742},"last":1.4730000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574958,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447392,"statfs":{"total":95995035648,"available":95690129408,"internally_reserved":0,"allocated":679936,"data_stored":245542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.57199999999999995},"min":{"1min":0.54200000000000004,"5min":0.41399999999999998,"15min":0.30499999999999999},"max":{"1min":1.0189999999999999,"5min":1.149,"15min":1.149},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.745,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":1.012,"15min":1.1359999999999999},"last":0.67700000000000005}]},{"osd":2,"last update":"Sun Feb 18 01:20:49 2024","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.77600000000000002,"15min":0.70399999999999996},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.999,"5min":1.113,"15min":1.113},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.752,"15min":0.69299999999999995},"min":{"1min":0.56499999999999995,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":0.95299999999999996,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.71799999999999997}]},{"osd":3,"last update":"Sun Feb 18 01:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1240000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.90100000000000002,"15min":0.84799999999999998},"min":{"1min":0.65200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.331,"5min":2.504,"15min":2.504},"last":0.84499999999999997}]},{"osd":4,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.93300000000000005,"15min":0.91000000000000003},"min":{"1min":0.67300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2829999999999999,"5min":1.96,"15min":1.96},"last":1.498},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.044,"15min":0.99399999999999999},"min":{"1min":0.76300000000000001,"5min":0.755,"15min":0.72599999999999998},"max":{"1min":1.393,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.371}]},{"osd":5,"last update":"Sun Feb 18 01:20:49 2024","interfaces":[{"interface":"back","average":{"1min":1.073,"5min":1.083,"15min":1.052},"min":{"1min":0.77600000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.361,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":1.3009999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.05,"15min":1.006},"min":{"1min":0.79100000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.694,"5min":1.855,"15min":1.855},"last":1.569}]},{"osd":6,"last update":"Sun Feb 18 01:21:24 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.099,"15min":1.091},"min":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3660000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.419},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1499999999999999,"15min":1.1319999999999999},"min":{"1min":0.877,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4990000000000001,"5min":1.698,"15min":1.698},"last":1.47}]},{"osd":7,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1679999999999999,"15min":1.1599999999999999},"min":{"1min":0.90900000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.4390000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.345},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.212,"15min":1.1910000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5589999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.552}]}]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":245241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.57799999999999996},"min":{"1min":0.52800000000000002,"5min":0.40100000000000002,"15min":0.38},"max":{"1min":0.92100000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.628},"min":{"1min":0.54200000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.73099999999999998}]},{"osd":2,"last update":"Sun Feb 18 01:20:46 2024","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.79600000000000004,"15min":0.69299999999999995},"min":{"1min":0.58599999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":1.02},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.73199999999999998},"min":{"1min":0.61799999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.089,"5min":1.149,"15min":1.149},"last":0.81699999999999995}]},{"osd":3,"last update":"Sun Feb 18 01:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88100000000000001,"15min":0.78700000000000003},"min":{"1min":0.68999999999999995,"5min":0.58599999999999997,"15min":0.51300000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":1.0740000000000001},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.85199999999999998},"min":{"1min":0.746,"5min":0.56000000000000005,"15min":0.55300000000000005},"max":{"1min":1.1659999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":1.0629999999999999}]},{"osd":4,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.002,"15min":0.93500000000000005},"min":{"1min":0.73499999999999999,"5min":0.71599999999999997,"15min":0.68500000000000005},"max":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.75700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1279999999999999,"5min":1.284,"15min":1.284},"last":0.89600000000000002}]},{"osd":5,"last update":"Sun Feb 18 01:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.099},"min":{"1min":0.79500000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.454}]},{"osd":6,"last update":"Sun Feb 18 01:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1259999999999999,"15min":1.1060000000000001},"min":{"1min":0.90000000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5549999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.125},{"interface":"front","average":{"1min":1.113,"5min":1.0580000000000001,"15min":1.0660000000000001},"min":{"1min":0.90600000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.403,"5min":1.444,"15min":1.444},"last":1.1479999999999999}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":1.046,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.548},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2070000000000001},"min":{"1min":0.995,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.5169999999999999,"5min":1.675,"15min":1.675},"last":1.194}]}]},{"osd":2,"up_from":18,"seq":77309411430,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297708,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447444,"statfs":{"total":95995035648,"available":95690182656,"internally_reserved":0,"allocated":626688,"data_stored":243844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70599999999999996,"15min":0.59199999999999997},"min":{"1min":0.56399999999999995,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.63900000000000001},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.32500000000000001},"max":{"1min":0.94099999999999995,"5min":1.224,"15min":1.224},"last":0.66800000000000004}]},{"osd":1,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.64300000000000002},"min":{"1min":0.42699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0289999999999999,"5min":1.361,"15min":1.4159999999999999},"last":0.72899999999999998}]},{"osd":3,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80500000000000005,"15min":0.76700000000000002},"min":{"1min":0.53800000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.84399999999999997},"min":{"1min":0.63,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.85399999999999998}]},{"osd":4,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0489999999999999,"15min":0.98999999999999999},"min":{"1min":0.74299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.94099999999999995},"min":{"1min":0.86199999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.24,"5min":1.393,"15min":1.393},"last":0.90300000000000002}]},{"osd":5,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.107,"15min":1.0569999999999999},"min":{"1min":0.86299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.321,"5min":1.2170000000000001,"15min":1.198},"min":{"1min":1.034,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.734},"last":0.95799999999999996}]},{"osd":6,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.1479999999999999},"min":{"1min":0.86199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.262,"5min":1.2210000000000001,"15min":1.2110000000000001},"min":{"1min":0.96599999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.631,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.262}]},{"osd":7,"last update":"Sun Feb 18 01:21:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.248,"15min":1.234},"min":{"1min":0.88,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1100000000000001,"15min":1.1060000000000001},"min":{"1min":0.875,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4299999999999999,"5min":1.502,"15min":1.502},"last":1.014}]}]},{"osd":3,"up_from":25,"seq":107374182493,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":245351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68799999999999994,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.747}]},{"osd":1,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.68799999999999994},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92300000000000004,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.39900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.077,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.48099999999999998}]},{"osd":2,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85999999999999999,"15min":0.82399999999999995},"min":{"1min":0.66200000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1559999999999999,"5min":1.3,"15min":1.3},"last":0.82099999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.155,"5min":1.27,"15min":1.286},"last":0.499}]},{"osd":4,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92900000000000005,"15min":0.91100000000000003},"min":{"1min":0.64900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.175,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.002,"15min":0.94499999999999995},"min":{"1min":0.81299999999999994,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.407,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":0.61799999999999999}]},{"osd":5,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0129999999999999,"15min":0.997},"min":{"1min":0.76900000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.70399999999999996},{"interface":"front","average":{"1min":1.034,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.75700000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3660000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":0.86099999999999999}]},{"osd":6,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0900000000000001,"15min":1.097},"min":{"1min":0.95599999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5820000000000001,"5min":1.625,"15min":1.625},"last":0.94599999999999995},{"interface":"front","average":{"1min":1.165,"5min":1.1100000000000001,"15min":1.0820000000000001},"min":{"1min":0.82499999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3560000000000001,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1100000000000001,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3540000000000001,"5min":1.625,"15min":1.625},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1379999999999999,"15min":1.1080000000000001},"min":{"1min":0.878,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6739999999999999,"5min":1.748,"15min":1.748},"last":0.90500000000000003}]}]},{"osd":4,"up_from":31,"seq":133143986260,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297740,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447412,"statfs":{"total":95995035648,"available":95690149888,"internally_reserved":0,"allocated":659456,"data_stored":245377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.90900000000000003,"15min":0.85999999999999999},"min":{"1min":0.58399999999999996,"5min":0.435,"15min":0.435},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":0.68999999999999995},{"interface":"front","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.86699999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.86699999999999999}]},{"osd":1,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91400000000000003,"15min":0.85499999999999998},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.872},"min":{"1min":0.745,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":1.0469999999999999}]},{"osd":2,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92900000000000005,"15min":0.85599999999999998},"min":{"1min":0.749,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2649999999999999,"5min":1.806,"15min":1.806},"last":0.79900000000000004},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.97299999999999998,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.97499999999999998}]},{"osd":3,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.96799999999999997,"15min":0.94099999999999995},"min":{"1min":0.79900000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98699999999999999,"15min":0.93999999999999995},"min":{"1min":0.76000000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":1.1259999999999999}]},{"osd":5,"last update":"Sun Feb 18 01:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92300000000000004,"15min":0.85199999999999998},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.3320000000000001,"5min":2.177,"15min":2.177},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.0860000000000001,"15min":0.95299999999999996},"min":{"1min":0.83899999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"max":{"1min":1.7030000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.1559999999999999}]},{"osd":6,"last update":"Sun Feb 18 01:21:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.036,"15min":1.0760000000000001},"min":{"1min":0.77800000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.1659999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.075,"5min":1.0229999999999999,"15min":1.0429999999999999},"min":{"1min":0.67100000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6479999999999999,"5min":2.508,"15min":2.508},"last":1.2070000000000001}]},{"osd":7,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0609999999999999,"15min":1.0209999999999999},"min":{"1min":0.754,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3859999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.073,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.79500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3260000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1870000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790029,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":243309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85699999999999998,"15min":0.82599999999999996},"min":{"1min":0.61699999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.21,"5min":1.248,"15min":1.248},"last":1.012},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.85599999999999998},"min":{"1min":0.66800000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0880000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.1839999999999999}]},{"osd":1,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.879,"15min":0.878},"min":{"1min":0.59299999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":1.232},{"interface":"front","average":{"1min":0.878,"5min":0.89400000000000002,"15min":0.84599999999999997},"min":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.83699999999999997}]},{"osd":2,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.90400000000000003,"15min":0.89700000000000002},"min":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.28,"15min":1.28},"last":1.054},{"interface":"front","average":{"1min":1,"5min":0.94499999999999995,"15min":0.88100000000000001},"min":{"1min":0.628,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.78300000000000003}]},{"osd":3,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.72099999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.046,"5min":1.0149999999999999,"15min":0.96899999999999997},"min":{"1min":0.83799999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.88300000000000001}]},{"osd":4,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88400000000000001,"15min":0.81200000000000006},"min":{"1min":0.625,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":1.1599999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.081},{"interface":"front","average":{"1min":1.087,"5min":1.032,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.52700000000000002},"max":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.121}]},{"osd":6,"last update":"Sun Feb 18 01:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.94399999999999995,"15min":0.91600000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.208},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.002,"15min":0.95299999999999996},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.411,"5min":1.423,"15min":1.423},"last":1.141}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.01,"15min":1.028},"min":{"1min":0.76600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.234,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.3640000000000001,"5min":1.371,"15min":1.371},"last":1.165}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T01:21:52.011 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph pg dump --format=json 2024-02-18T01:21:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:21:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:21:52.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:52 smithi105 bash[27669]: audit 2024-02-18T01:21:51.306034+0000 mgr.y (mgr.24475) 181 : audit [DBG] from='client.24674 v1:172.21.15.105:0/876789531' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:21:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:52 smithi105 bash[19734]: audit 2024-02-18T01:21:51.306034+0000 mgr.y (mgr.24475) 181 : audit [DBG] from='client.24674 v1:172.21.15.105:0/876789531' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:21:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:52 smithi155 bash[20805]: audit 2024-02-18T01:21:51.306034+0000 mgr.y (mgr.24475) 181 : audit [DBG] from='client.24674 v1:172.21.15.105:0/876789531' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:21:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:53 smithi105 bash[19734]: cluster 2024-02-18T01:21:52.384033+0000 mgr.y (mgr.24475) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:53.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:53 smithi105 bash[27669]: cluster 2024-02-18T01:21:52.384033+0000 mgr.y (mgr.24475) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:53.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:53 smithi155 bash[20805]: cluster 2024-02-18T01:21:52.384033+0000 mgr.y (mgr.24475) 182 : cluster [DBG] pgmap v133: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:55 smithi105 bash[19734]: cluster 2024-02-18T01:21:54.384772+0000 mgr.y (mgr.24475) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:55 smithi105 bash[27669]: cluster 2024-02-18T01:21:54.384772+0000 mgr.y (mgr.24475) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:55.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:55 smithi155 bash[20805]: cluster 2024-02-18T01:21:54.384772+0000 mgr.y (mgr.24475) 183 : cluster [DBG] pgmap v134: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:56.850 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:21:56.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:21:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:21:56] "GET /metrics HTTP/1.1" 200 172175 "" "Prometheus/2.33.4" 2024-02-18T01:21:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:57 smithi105 bash[19734]: cluster 2024-02-18T01:21:56.385554+0000 mgr.y (mgr.24475) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:57.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:57 smithi105 bash[27669]: cluster 2024-02-18T01:21:56.385554+0000 mgr.y (mgr.24475) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:57.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:57 smithi155 bash[20805]: cluster 2024-02-18T01:21:56.385554+0000 mgr.y (mgr.24475) 184 : cluster [DBG] pgmap v135: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:21:57.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:21:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:21:58.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:58 smithi105 bash[19734]: audit 2024-02-18T01:21:57.508566+0000 mgr.y (mgr.24475) 185 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:58 smithi105 bash[27669]: audit 2024-02-18T01:21:57.508566+0000 mgr.y (mgr.24475) 185 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:58 smithi155 bash[20805]: audit 2024-02-18T01:21:57.508566+0000 mgr.y (mgr.24475) 185 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:21:59.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:21:59 smithi105 bash[19734]: cluster 2024-02-18T01:21:58.387037+0000 mgr.y (mgr.24475) 186 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:21:59 smithi105 bash[27669]: cluster 2024-02-18T01:21:58.387037+0000 mgr.y (mgr.24475) 186 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.734+0000 7f095766d700 1 -- 172.21.15.105:0/498140288 >> v1:172.21.15.105:6789/0 conn(0x7f09501032b0 legacy=0x7f0950105730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.734+0000 7f095766d700 1 -- 172.21.15.105:0/498140288 shutdown_connections 2024-02-18T01:21:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.734+0000 7f095766d700 1 -- 172.21.15.105:0/498140288 >> 172.21.15.105:0/498140288 conn(0x7f09500f2530 msgr2=0x7f09500f4950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.734+0000 7f095766d700 1 -- 172.21.15.105:0/498140288 shutdown_connections 2024-02-18T01:21:59.741 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.734+0000 7f095766d700 1 -- 172.21.15.105:0/498140288 wait complete. 2024-02-18T01:21:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 Processor -- start 2024-02-18T01:21:59.742 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- start start 2024-02-18T01:21:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09501167f0 con 0x7f09501032b0 2024-02-18T01:21:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0950117990 con 0x7f09500fc970 2024-02-18T01:21:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f0954c08700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f09500ff770 0x7f0950111910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:49892/0 (socket says 172.21.15.105:49892) 2024-02-18T01:21:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f0950118b70 con 0x7f09500ff770 2024-02-18T01:21:59.743 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f0954c08700 1 -- 172.21.15.105:0/74174165 learned_addr learned my addr 172.21.15.105:0/74174165 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:21:59.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:21:59 smithi155 bash[20805]: cluster 2024-02-18T01:21:58.387037+0000 mgr.y (mgr.24475) 186 : cluster [DBG] pgmap v136: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276114839 0 0) 0x7f0950118b70 con 0x7f09500ff770 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f093c003630 con 0x7f09500ff770 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462704061 0 0) 0x7f09501167f0 con 0x7f09501032b0 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0950118b70 con 0x7f09501032b0 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 597153687 0 0) 0x7f093c003630 con 0x7f09500ff770 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09501167f0 con 0x7f09500ff770 2024-02-18T01:21:59.745 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2366126863 0 0) 0x7f0950118b70 con 0x7f09501032b0 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f093c003630 con 0x7f09501032b0 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0940002900 con 0x7f09500ff770 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0948002fe0 con 0x7f09501032b0 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2376327499 0 0) 0x7f0950117990 con 0x7f09500fc970 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0950118b70 con 0x7f09500fc970 2024-02-18T01:21:59.746 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2562843762 0 0) 0x7f09501167f0 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 >> v1:172.21.15.155:6789/0 conn(0x7f09500fc970 legacy=0x7f09500fb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 >> v1:172.21.15.105:6789/0 conn(0x7f09501032b0 legacy=0x7f0950115040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0950060b70 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0950060e20 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f0950061290 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f0940001740 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.738+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f0940005ac0 con 0x7f09500ff770 2024-02-18T01:21:59.747 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.742+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0934004f20 con 0x7f09500ff770 2024-02-18T01:21:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.742+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f09400012a0 con 0x7f09500ff770 2024-02-18T01:21:59.748 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.742+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f0940078590 con 0x7f09500ff770 2024-02-18T01:21:59.754 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.750+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f094004c100 con 0x7f09500ff770 2024-02-18T01:21:59.936 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.930+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 --> v1:172.21.15.105:6800/2148176012 -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0934000b90 con 0x7f093c0623e0 2024-02-18T01:21:59.957 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.950+0000 7f09467fc700 1 -- 172.21.15.105:0/74174165 <== mgr.24475 v1:172.21.15.105:6800/2148176012 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+355230 (unknown 2965378022 0 2817774332) 0x7f0934000b90 con 0x7f093c0623e0 2024-02-18T01:21:59.962 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 >> v1:172.21.15.105:6800/2148176012 conn(0x7f093c0623e0 legacy=0x7f093c0648a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 >> v1:172.21.15.105:6790/0 conn(0x7f09500ff770 legacy=0x7f0950111910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 shutdown_connections 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 >> 172.21.15.105:0/74174165 conn(0x7f09500f2530 msgr2=0x7f0950106720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 shutdown_connections 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:21:59.958+0000 7f095766d700 1 -- 172.21.15.105:0/74174165 wait complete. 2024-02-18T01:21:59.965 INFO:teuthology.orchestra.run.smithi105.stderr:dumped all 2024-02-18T01:22:00.594 INFO:teuthology.orchestra.run.smithi105.stdout:{"pg_ready":true,"pg_map":{"version":136,"stamp":"2024-02-18T01:21:58.385879+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":5796,"num_objects":224,"num_object_clones":0,"num_object_copies":672,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":224,"num_whiteouts":0,"num_read":1383,"num_read_kb":1103,"num_write":620,"num_write_kb":41,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":660,"ondisk_log_size":660,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2381824,"kb_used_data":5056,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":747579392,"statfs":{"total":767960285184,"available":765521297408,"internally_reserved":0,"allocated":5177344,"data_stored":1956979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005870"},"pg_stats":[{"pgid":"6.1a","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544813+0000","last_change":"2024-02-18T01:17:16.835984+0000","last_active":"2024-02-18T01:18:36.544813+0000","last_peered":"2024-02-18T01:18:36.544813+0000","last_clean":"2024-02-18T01:18:36.544813+0000","last_became_active":"2024-02-18T01:17:16.835727+0000","last_became_peered":"2024-02-18T01:17:16.835727+0000","last_unstale":"2024-02-18T01:18:36.544813+0000","last_undegraded":"2024-02-18T01:18:36.544813+0000","last_fullsized":"2024-02-18T01:18:36.544813+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543726+0000","last_change":"2024-02-18T01:17:14.832757+0000","last_active":"2024-02-18T01:18:36.543726+0000","last_peered":"2024-02-18T01:18:36.543726+0000","last_clean":"2024-02-18T01:18:36.543726+0000","last_became_active":"2024-02-18T01:17:14.832433+0000","last_became_peered":"2024-02-18T01:17:14.832433+0000","last_unstale":"2024-02-18T01:18:36.543726+0000","last_undegraded":"2024-02-18T01:18:36.543726+0000","last_fullsized":"2024-02-18T01:18:36.543726+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551995+0000","last_change":"2024-02-18T01:17:11.299424+0000","last_active":"2024-02-18T01:18:36.551995+0000","last_peered":"2024-02-18T01:18:36.551995+0000","last_clean":"2024-02-18T01:18:36.551995+0000","last_became_active":"2024-02-18T01:17:11.298491+0000","last_became_peered":"2024-02-18T01:17:11.298491+0000","last_unstale":"2024-02-18T01:18:36.551995+0000","last_undegraded":"2024-02-18T01:18:36.551995+0000","last_fullsized":"2024-02-18T01:18:36.551995+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"61'15","reported_seq":69,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.568556+0000","last_change":"2024-02-18T01:18:37.568556+0000","last_active":"2024-02-18T01:18:37.568556+0000","last_peered":"2024-02-18T01:18:37.568556+0000","last_clean":"2024-02-18T01:18:37.568556+0000","last_became_active":"2024-02-18T01:18:37.559301+0000","last_became_peered":"2024-02-18T01:18:37.559301+0000","last_unstale":"2024-02-18T01:18:37.568556+0000","last_undegraded":"2024-02-18T01:18:37.568556+0000","last_fullsized":"2024-02-18T01:18:37.568556+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,2],"acting":[4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586026+0000","last_change":"2024-02-18T01:17:17.095065+0000","last_active":"2024-02-18T01:18:36.586026+0000","last_peered":"2024-02-18T01:18:36.586026+0000","last_clean":"2024-02-18T01:18:36.586026+0000","last_became_active":"2024-02-18T01:17:17.094717+0000","last_became_peered":"2024-02-18T01:17:17.094717+0000","last_unstale":"2024-02-18T01:18:36.586026+0000","last_undegraded":"2024-02-18T01:18:36.586026+0000","last_fullsized":"2024-02-18T01:18:36.586026+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544919+0000","last_change":"2024-02-18T01:17:14.834664+0000","last_active":"2024-02-18T01:18:36.544919+0000","last_peered":"2024-02-18T01:18:36.544919+0000","last_clean":"2024-02-18T01:18:36.544919+0000","last_became_active":"2024-02-18T01:17:14.834270+0000","last_became_peered":"2024-02-18T01:17:14.834270+0000","last_unstale":"2024-02-18T01:18:36.544919+0000","last_undegraded":"2024-02-18T01:18:36.544919+0000","last_fullsized":"2024-02-18T01:18:36.544919+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586963+0000","last_change":"2024-02-18T01:17:11.298980+0000","last_active":"2024-02-18T01:18:36.586963+0000","last_peered":"2024-02-18T01:18:36.586963+0000","last_clean":"2024-02-18T01:18:36.586963+0000","last_became_active":"2024-02-18T01:17:11.298577+0000","last_became_peered":"2024-02-18T01:17:11.298577+0000","last_unstale":"2024-02-18T01:18:36.586963+0000","last_undegraded":"2024-02-18T01:18:36.586963+0000","last_fullsized":"2024-02-18T01:18:36.586963+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"61'39","reported_seq":102,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586084+0000","last_change":"2024-02-18T01:17:12.825782+0000","last_active":"2024-02-18T01:18:36.586084+0000","last_peered":"2024-02-18T01:18:36.586084+0000","last_clean":"2024-02-18T01:18:36.586084+0000","last_became_active":"2024-02-18T01:17:12.824964+0000","last_became_peered":"2024-02-18T01:17:12.824964+0000","last_unstale":"2024-02-18T01:18:36.586084+0000","last_undegraded":"2024-02-18T01:18:36.586084+0000","last_fullsized":"2024-02-18T01:18:36.586084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":39,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":40,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549076+0000","last_change":"2024-02-18T01:17:16.835292+0000","last_active":"2024-02-18T01:18:36.549076+0000","last_peered":"2024-02-18T01:18:36.549076+0000","last_clean":"2024-02-18T01:18:36.549076+0000","last_became_active":"2024-02-18T01:17:16.834962+0000","last_became_peered":"2024-02-18T01:17:16.834962+0000","last_unstale":"2024-02-18T01:18:36.549076+0000","last_undegraded":"2024-02-18T01:18:36.549076+0000","last_fullsized":"2024-02-18T01:18:36.549076+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548967+0000","last_change":"2024-02-18T01:17:14.832828+0000","last_active":"2024-02-18T01:18:36.548967+0000","last_peered":"2024-02-18T01:18:36.548967+0000","last_clean":"2024-02-18T01:18:36.548967+0000","last_became_active":"2024-02-18T01:17:14.832503+0000","last_became_peered":"2024-02-18T01:17:14.832503+0000","last_unstale":"2024-02-18T01:18:36.548967+0000","last_undegraded":"2024-02-18T01:18:36.548967+0000","last_fullsized":"2024-02-18T01:18:36.548967+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549785+0000","last_change":"2024-02-18T01:17:10.705677+0000","last_active":"2024-02-18T01:18:36.549785+0000","last_peered":"2024-02-18T01:18:36.549785+0000","last_clean":"2024-02-18T01:18:36.549785+0000","last_became_active":"2024-02-18T01:17:10.705303+0000","last_became_peered":"2024-02-18T01:17:10.705303+0000","last_unstale":"2024-02-18T01:18:36.549785+0000","last_undegraded":"2024-02-18T01:18:36.549785+0000","last_fullsized":"2024-02-18T01:18:36.549785+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"61'21","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545409+0000","last_change":"2024-02-18T01:17:12.825755+0000","last_active":"2024-02-18T01:18:36.545409+0000","last_peered":"2024-02-18T01:18:36.545409+0000","last_clean":"2024-02-18T01:18:36.545409+0000","last_became_active":"2024-02-18T01:17:12.825377+0000","last_became_peered":"2024-02-18T01:17:12.825377+0000","last_unstale":"2024-02-18T01:18:36.545409+0000","last_undegraded":"2024-02-18T01:18:36.545409+0000","last_fullsized":"2024-02-18T01:18:36.545409+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548406+0000","last_change":"2024-02-18T01:17:16.839543+0000","last_active":"2024-02-18T01:18:36.548406+0000","last_peered":"2024-02-18T01:18:36.548406+0000","last_clean":"2024-02-18T01:18:36.548406+0000","last_became_active":"2024-02-18T01:17:16.839311+0000","last_became_peered":"2024-02-18T01:17:16.839311+0000","last_unstale":"2024-02-18T01:18:36.548406+0000","last_undegraded":"2024-02-18T01:18:36.548406+0000","last_fullsized":"2024-02-18T01:18:36.548406+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494362+0000","last_change":"2024-02-18T01:17:14.827214+0000","last_active":"2024-02-18T01:18:37.494362+0000","last_peered":"2024-02-18T01:18:37.494362+0000","last_clean":"2024-02-18T01:18:37.494362+0000","last_became_active":"2024-02-18T01:17:14.826950+0000","last_became_peered":"2024-02-18T01:17:14.826950+0000","last_unstale":"2024-02-18T01:18:37.494362+0000","last_undegraded":"2024-02-18T01:18:37.494362+0000","last_fullsized":"2024-02-18T01:18:37.494362+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549045+0000","last_change":"2024-02-18T01:17:10.706739+0000","last_active":"2024-02-18T01:18:36.549045+0000","last_peered":"2024-02-18T01:18:36.549045+0000","last_clean":"2024-02-18T01:18:36.549045+0000","last_became_active":"2024-02-18T01:17:10.706172+0000","last_became_peered":"2024-02-18T01:17:10.706172+0000","last_unstale":"2024-02-18T01:18:36.549045+0000","last_undegraded":"2024-02-18T01:18:36.549045+0000","last_fullsized":"2024-02-18T01:18:36.549045+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"61'11","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545238+0000","last_change":"2024-02-18T01:17:12.826222+0000","last_active":"2024-02-18T01:18:36.545238+0000","last_peered":"2024-02-18T01:18:36.545238+0000","last_clean":"2024-02-18T01:18:36.545238+0000","last_became_active":"2024-02-18T01:17:12.825893+0000","last_became_peered":"2024-02-18T01:17:12.825893+0000","last_unstale":"2024-02-18T01:18:36.545238+0000","last_undegraded":"2024-02-18T01:18:36.545238+0000","last_fullsized":"2024-02-18T01:18:36.545238+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":23,"num_read_kb":18,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546527+0000","last_change":"2024-02-18T01:17:17.095339+0000","last_active":"2024-02-18T01:18:36.546527+0000","last_peered":"2024-02-18T01:18:36.546527+0000","last_clean":"2024-02-18T01:18:36.546527+0000","last_became_active":"2024-02-18T01:17:17.094461+0000","last_became_peered":"2024-02-18T01:17:17.094461+0000","last_unstale":"2024-02-18T01:18:36.546527+0000","last_undegraded":"2024-02-18T01:18:36.546527+0000","last_fullsized":"2024-02-18T01:18:36.546527+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547559+0000","last_change":"2024-02-18T01:17:14.828698+0000","last_active":"2024-02-18T01:18:36.547559+0000","last_peered":"2024-02-18T01:18:36.547559+0000","last_clean":"2024-02-18T01:18:36.547559+0000","last_became_active":"2024-02-18T01:17:14.828373+0000","last_became_peered":"2024-02-18T01:17:14.828373+0000","last_unstale":"2024-02-18T01:18:36.547559+0000","last_undegraded":"2024-02-18T01:18:36.547559+0000","last_fullsized":"2024-02-18T01:18:36.547559+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549938+0000","last_change":"2024-02-18T01:17:10.711503+0000","last_active":"2024-02-18T01:18:36.549938+0000","last_peered":"2024-02-18T01:18:36.549938+0000","last_clean":"2024-02-18T01:18:36.549938+0000","last_became_active":"2024-02-18T01:17:10.711207+0000","last_became_peered":"2024-02-18T01:17:10.711207+0000","last_unstale":"2024-02-18T01:18:36.549938+0000","last_undegraded":"2024-02-18T01:18:36.549938+0000","last_fullsized":"2024-02-18T01:18:36.549938+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"61'15","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545594+0000","last_change":"2024-02-18T01:17:12.821480+0000","last_active":"2024-02-18T01:18:36.545594+0000","last_peered":"2024-02-18T01:18:36.545594+0000","last_clean":"2024-02-18T01:18:36.545594+0000","last_became_active":"2024-02-18T01:17:12.821084+0000","last_became_peered":"2024-02-18T01:17:12.821084+0000","last_unstale":"2024-02-18T01:18:36.545594+0000","last_undegraded":"2024-02-18T01:18:36.545594+0000","last_fullsized":"2024-02-18T01:18:36.545594+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586399+0000","last_change":"2024-02-18T01:17:17.093600+0000","last_active":"2024-02-18T01:18:36.586399+0000","last_peered":"2024-02-18T01:18:36.586399+0000","last_clean":"2024-02-18T01:18:36.586399+0000","last_became_active":"2024-02-18T01:17:17.092661+0000","last_became_peered":"2024-02-18T01:17:17.092661+0000","last_unstale":"2024-02-18T01:18:36.586399+0000","last_undegraded":"2024-02-18T01:18:36.586399+0000","last_fullsized":"2024-02-18T01:18:36.586399+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545322+0000","last_change":"2024-02-18T01:17:14.823942+0000","last_active":"2024-02-18T01:18:36.545322+0000","last_peered":"2024-02-18T01:18:36.545322+0000","last_clean":"2024-02-18T01:18:36.545322+0000","last_became_active":"2024-02-18T01:17:14.823204+0000","last_became_peered":"2024-02-18T01:17:14.823204+0000","last_unstale":"2024-02-18T01:18:36.545322+0000","last_undegraded":"2024-02-18T01:18:36.545322+0000","last_fullsized":"2024-02-18T01:18:36.545322+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545528+0000","last_change":"2024-02-18T01:17:11.301098+0000","last_active":"2024-02-18T01:18:36.545528+0000","last_peered":"2024-02-18T01:18:36.545528+0000","last_clean":"2024-02-18T01:18:36.545528+0000","last_became_active":"2024-02-18T01:17:11.300829+0000","last_became_peered":"2024-02-18T01:17:11.300829+0000","last_unstale":"2024-02-18T01:18:36.545528+0000","last_undegraded":"2024-02-18T01:18:36.545528+0000","last_fullsized":"2024-02-18T01:18:36.545528+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"61'18","reported_seq":55,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586842+0000","last_change":"2024-02-18T01:17:12.830495+0000","last_active":"2024-02-18T01:18:36.586842+0000","last_peered":"2024-02-18T01:18:36.586842+0000","last_clean":"2024-02-18T01:18:36.586842+0000","last_became_active":"2024-02-18T01:17:12.830247+0000","last_became_peered":"2024-02-18T01:17:12.830247+0000","last_unstale":"2024-02-18T01:18:36.586842+0000","last_undegraded":"2024-02-18T01:18:36.586842+0000","last_fullsized":"2024-02-18T01:18:36.586842+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":18,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":37,"num_read_kb":27,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"61'1","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493123+0000","last_change":"2024-02-18T01:17:16.839142+0000","last_active":"2024-02-18T01:18:37.493123+0000","last_peered":"2024-02-18T01:18:37.493123+0000","last_clean":"2024-02-18T01:18:37.493123+0000","last_became_active":"2024-02-18T01:17:16.838801+0000","last_became_peered":"2024-02-18T01:17:16.838801+0000","last_unstale":"2024-02-18T01:18:37.493123+0000","last_undegraded":"2024-02-18T01:18:37.493123+0000","last_fullsized":"2024-02-18T01:18:37.493123+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"61'8","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142694+0000","last_change":"2024-02-18T01:17:14.838597+0000","last_active":"2024-02-18T01:21:49.142694+0000","last_peered":"2024-02-18T01:21:49.142694+0000","last_clean":"2024-02-18T01:21:49.142694+0000","last_became_active":"2024-02-18T01:17:14.836578+0000","last_became_peered":"2024-02-18T01:17:14.836578+0000","last_unstale":"2024-02-18T01:21:49.142694+0000","last_undegraded":"2024-02-18T01:21:49.142694+0000","last_fullsized":"2024-02-18T01:21:49.142694+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544050+0000","last_change":"2024-02-18T01:17:10.704953+0000","last_active":"2024-02-18T01:18:36.544050+0000","last_peered":"2024-02-18T01:18:36.544050+0000","last_clean":"2024-02-18T01:18:36.544050+0000","last_became_active":"2024-02-18T01:17:10.704586+0000","last_became_peered":"2024-02-18T01:17:10.704586+0000","last_unstale":"2024-02-18T01:18:36.544050+0000","last_undegraded":"2024-02-18T01:18:36.544050+0000","last_fullsized":"2024-02-18T01:18:36.544050+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"61'16","reported_seq":47,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551572+0000","last_change":"2024-02-18T01:17:12.823321+0000","last_active":"2024-02-18T01:18:36.551572+0000","last_peered":"2024-02-18T01:18:36.551572+0000","last_clean":"2024-02-18T01:18:36.551572+0000","last_became_active":"2024-02-18T01:17:12.822982+0000","last_became_peered":"2024-02-18T01:17:12.822982+0000","last_unstale":"2024-02-18T01:18:36.551572+0000","last_undegraded":"2024-02-18T01:18:36.551572+0000","last_fullsized":"2024-02-18T01:18:36.551572+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547651+0000","last_change":"2024-02-18T01:17:16.835943+0000","last_active":"2024-02-18T01:18:36.547651+0000","last_peered":"2024-02-18T01:18:36.547651+0000","last_clean":"2024-02-18T01:18:36.547651+0000","last_became_active":"2024-02-18T01:17:16.835626+0000","last_became_peered":"2024-02-18T01:17:16.835626+0000","last_unstale":"2024-02-18T01:18:36.547651+0000","last_undegraded":"2024-02-18T01:18:36.547651+0000","last_fullsized":"2024-02-18T01:18:36.547651+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551822+0000","last_change":"2024-02-18T01:17:14.829269+0000","last_active":"2024-02-18T01:18:36.551822+0000","last_peered":"2024-02-18T01:18:36.551822+0000","last_clean":"2024-02-18T01:18:36.551822+0000","last_became_active":"2024-02-18T01:17:14.828815+0000","last_became_peered":"2024-02-18T01:17:14.828815+0000","last_unstale":"2024-02-18T01:18:36.551822+0000","last_undegraded":"2024-02-18T01:18:36.551822+0000","last_fullsized":"2024-02-18T01:18:36.551822+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585385+0000","last_change":"2024-02-18T01:17:10.706768+0000","last_active":"2024-02-18T01:18:36.585385+0000","last_peered":"2024-02-18T01:18:36.585385+0000","last_clean":"2024-02-18T01:18:36.585385+0000","last_became_active":"2024-02-18T01:17:10.706273+0000","last_became_peered":"2024-02-18T01:17:10.706273+0000","last_unstale":"2024-02-18T01:18:36.585385+0000","last_undegraded":"2024-02-18T01:18:36.585385+0000","last_fullsized":"2024-02-18T01:18:36.585385+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549489+0000","last_change":"2024-02-18T01:17:12.829234+0000","last_active":"2024-02-18T01:18:36.549489+0000","last_peered":"2024-02-18T01:18:36.549489+0000","last_clean":"2024-02-18T01:18:36.549489+0000","last_became_active":"2024-02-18T01:17:12.828666+0000","last_became_peered":"2024-02-18T01:17:12.828666+0000","last_unstale":"2024-02-18T01:18:36.549489+0000","last_undegraded":"2024-02-18T01:18:36.549489+0000","last_fullsized":"2024-02-18T01:18:36.549489+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.492998+0000","last_change":"2024-02-18T01:17:16.838484+0000","last_active":"2024-02-18T01:18:37.492998+0000","last_peered":"2024-02-18T01:18:37.492998+0000","last_clean":"2024-02-18T01:18:37.492998+0000","last_became_active":"2024-02-18T01:17:16.838194+0000","last_became_peered":"2024-02-18T01:17:16.838194+0000","last_unstale":"2024-02-18T01:18:37.492998+0000","last_undegraded":"2024-02-18T01:18:37.492998+0000","last_fullsized":"2024-02-18T01:18:37.492998+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547643+0000","last_change":"2024-02-18T01:17:14.836977+0000","last_active":"2024-02-18T01:18:36.547643+0000","last_peered":"2024-02-18T01:18:36.547643+0000","last_clean":"2024-02-18T01:18:36.547643+0000","last_became_active":"2024-02-18T01:17:14.836653+0000","last_became_peered":"2024-02-18T01:17:14.836653+0000","last_unstale":"2024-02-18T01:18:36.547643+0000","last_undegraded":"2024-02-18T01:18:36.547643+0000","last_fullsized":"2024-02-18T01:18:36.547643+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.552014+0000","last_change":"2024-02-18T01:17:10.705385+0000","last_active":"2024-02-18T01:18:36.552014+0000","last_peered":"2024-02-18T01:18:36.552014+0000","last_clean":"2024-02-18T01:18:36.552014+0000","last_became_active":"2024-02-18T01:17:10.705059+0000","last_became_peered":"2024-02-18T01:17:10.705059+0000","last_unstale":"2024-02-18T01:18:36.552014+0000","last_undegraded":"2024-02-18T01:18:36.552014+0000","last_fullsized":"2024-02-18T01:18:36.552014+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"61'4","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585445+0000","last_change":"2024-02-18T01:17:12.828867+0000","last_active":"2024-02-18T01:18:36.585445+0000","last_peered":"2024-02-18T01:18:36.585445+0000","last_clean":"2024-02-18T01:18:36.585445+0000","last_became_active":"2024-02-18T01:17:12.828557+0000","last_became_peered":"2024-02-18T01:17:12.828557+0000","last_unstale":"2024-02-18T01:18:36.585445+0000","last_undegraded":"2024-02-18T01:18:36.585445+0000","last_fullsized":"2024-02-18T01:18:36.585445+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585910+0000","last_change":"2024-02-18T01:17:17.093382+0000","last_active":"2024-02-18T01:18:36.585910+0000","last_peered":"2024-02-18T01:18:36.585910+0000","last_clean":"2024-02-18T01:18:36.585910+0000","last_became_active":"2024-02-18T01:17:17.092998+0000","last_became_peered":"2024-02-18T01:17:17.092998+0000","last_unstale":"2024-02-18T01:18:36.585910+0000","last_undegraded":"2024-02-18T01:18:36.585910+0000","last_fullsized":"2024-02-18T01:18:36.585910+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494868+0000","last_change":"2024-02-18T01:17:14.831650+0000","last_active":"2024-02-18T01:18:37.494868+0000","last_peered":"2024-02-18T01:18:37.494868+0000","last_clean":"2024-02-18T01:18:37.494868+0000","last_became_active":"2024-02-18T01:17:14.831382+0000","last_became_peered":"2024-02-18T01:17:14.831382+0000","last_unstale":"2024-02-18T01:18:37.494868+0000","last_undegraded":"2024-02-18T01:18:37.494868+0000","last_fullsized":"2024-02-18T01:18:37.494868+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549574+0000","last_change":"2024-02-18T01:17:10.710694+0000","last_active":"2024-02-18T01:18:36.549574+0000","last_peered":"2024-02-18T01:18:36.549574+0000","last_clean":"2024-02-18T01:18:36.549574+0000","last_became_active":"2024-02-18T01:17:10.710111+0000","last_became_peered":"2024-02-18T01:17:10.710111+0000","last_unstale":"2024-02-18T01:18:36.549574+0000","last_undegraded":"2024-02-18T01:18:36.549574+0000","last_fullsized":"2024-02-18T01:18:36.549574+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586025+0000","last_change":"2024-02-18T01:17:12.823207+0000","last_active":"2024-02-18T01:18:36.586025+0000","last_peered":"2024-02-18T01:18:36.586025+0000","last_clean":"2024-02-18T01:18:36.586025+0000","last_became_active":"2024-02-18T01:17:12.822888+0000","last_became_peered":"2024-02-18T01:17:12.822888+0000","last_unstale":"2024-02-18T01:18:36.586025+0000","last_undegraded":"2024-02-18T01:18:36.586025+0000","last_fullsized":"2024-02-18T01:18:36.586025+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548636+0000","last_change":"2024-02-18T01:17:16.834121+0000","last_active":"2024-02-18T01:18:36.548636+0000","last_peered":"2024-02-18T01:18:36.548636+0000","last_clean":"2024-02-18T01:18:36.548636+0000","last_became_active":"2024-02-18T01:17:16.833789+0000","last_became_peered":"2024-02-18T01:17:16.833789+0000","last_unstale":"2024-02-18T01:18:36.548636+0000","last_undegraded":"2024-02-18T01:18:36.548636+0000","last_fullsized":"2024-02-18T01:18:36.548636+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586173+0000","last_change":"2024-02-18T01:17:14.830922+0000","last_active":"2024-02-18T01:18:36.586173+0000","last_peered":"2024-02-18T01:18:36.586173+0000","last_clean":"2024-02-18T01:18:36.586173+0000","last_became_active":"2024-02-18T01:17:14.830580+0000","last_became_peered":"2024-02-18T01:17:14.830580+0000","last_unstale":"2024-02-18T01:18:36.586173+0000","last_undegraded":"2024-02-18T01:18:36.586173+0000","last_fullsized":"2024-02-18T01:18:36.586173+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494872+0000","last_change":"2024-02-18T01:17:10.716040+0000","last_active":"2024-02-18T01:18:37.494872+0000","last_peered":"2024-02-18T01:18:37.494872+0000","last_clean":"2024-02-18T01:18:37.494872+0000","last_became_active":"2024-02-18T01:17:10.715775+0000","last_became_peered":"2024-02-18T01:17:10.715775+0000","last_unstale":"2024-02-18T01:18:37.494872+0000","last_undegraded":"2024-02-18T01:18:37.494872+0000","last_fullsized":"2024-02-18T01:18:37.494872+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"61'9","reported_seq":33,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544692+0000","last_change":"2024-02-18T01:17:12.824681+0000","last_active":"2024-02-18T01:18:36.544692+0000","last_peered":"2024-02-18T01:18:36.544692+0000","last_clean":"2024-02-18T01:18:36.544692+0000","last_became_active":"2024-02-18T01:17:12.824381+0000","last_became_peered":"2024-02-18T01:17:12.824381+0000","last_unstale":"2024-02-18T01:18:36.544692+0000","last_undegraded":"2024-02-18T01:18:36.544692+0000","last_fullsized":"2024-02-18T01:18:36.544692+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":9,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585683+0000","last_change":"2024-02-18T01:17:16.841341+0000","last_active":"2024-02-18T01:18:36.585683+0000","last_peered":"2024-02-18T01:18:36.585683+0000","last_clean":"2024-02-18T01:18:36.585683+0000","last_became_active":"2024-02-18T01:17:16.840941+0000","last_became_peered":"2024-02-18T01:17:16.840941+0000","last_unstale":"2024-02-18T01:18:36.585683+0000","last_undegraded":"2024-02-18T01:18:36.585683+0000","last_fullsized":"2024-02-18T01:18:36.585683+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544049+0000","last_change":"2024-02-18T01:17:14.834100+0000","last_active":"2024-02-18T01:18:36.544049+0000","last_peered":"2024-02-18T01:18:36.544049+0000","last_clean":"2024-02-18T01:18:36.544049+0000","last_became_active":"2024-02-18T01:17:14.833849+0000","last_became_peered":"2024-02-18T01:17:14.833849+0000","last_unstale":"2024-02-18T01:18:36.544049+0000","last_undegraded":"2024-02-18T01:18:36.544049+0000","last_fullsized":"2024-02-18T01:18:36.544049+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545601+0000","last_change":"2024-02-18T01:17:10.714192+0000","last_active":"2024-02-18T01:18:36.545601+0000","last_peered":"2024-02-18T01:18:36.545601+0000","last_clean":"2024-02-18T01:18:36.545601+0000","last_became_active":"2024-02-18T01:17:10.713903+0000","last_became_peered":"2024-02-18T01:17:10.713903+0000","last_unstale":"2024-02-18T01:18:36.545601+0000","last_undegraded":"2024-02-18T01:18:36.545601+0000","last_fullsized":"2024-02-18T01:18:36.545601+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"61'11","reported_seq":36,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545580+0000","last_change":"2024-02-18T01:17:12.820899+0000","last_active":"2024-02-18T01:18:36.545580+0000","last_peered":"2024-02-18T01:18:36.545580+0000","last_clean":"2024-02-18T01:18:36.545580+0000","last_became_active":"2024-02-18T01:17:12.820556+0000","last_became_peered":"2024-02-18T01:17:12.820556+0000","last_unstale":"2024-02-18T01:18:36.545580+0000","last_undegraded":"2024-02-18T01:18:36.545580+0000","last_fullsized":"2024-02-18T01:18:36.545580+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":11,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551421+0000","last_change":"2024-02-18T01:17:16.840097+0000","last_active":"2024-02-18T01:18:36.551421+0000","last_peered":"2024-02-18T01:18:36.551421+0000","last_clean":"2024-02-18T01:18:36.551421+0000","last_became_active":"2024-02-18T01:17:16.839779+0000","last_became_peered":"2024-02-18T01:17:16.839779+0000","last_unstale":"2024-02-18T01:18:36.551421+0000","last_undegraded":"2024-02-18T01:18:36.551421+0000","last_fullsized":"2024-02-18T01:18:36.551421+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"61'8","reported_seq":82,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142586+0000","last_change":"2024-02-18T01:17:14.838558+0000","last_active":"2024-02-18T01:21:49.142586+0000","last_peered":"2024-02-18T01:21:49.142586+0000","last_clean":"2024-02-18T01:21:49.142586+0000","last_became_active":"2024-02-18T01:17:14.838236+0000","last_became_peered":"2024-02-18T01:17:14.838236+0000","last_unstale":"2024-02-18T01:21:49.142586+0000","last_undegraded":"2024-02-18T01:21:49.142586+0000","last_fullsized":"2024-02-18T01:21:49.142586+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493811+0000","last_change":"2024-02-18T01:17:11.299796+0000","last_active":"2024-02-18T01:18:37.493811+0000","last_peered":"2024-02-18T01:18:37.493811+0000","last_clean":"2024-02-18T01:18:37.493811+0000","last_became_active":"2024-02-18T01:17:11.299257+0000","last_became_peered":"2024-02-18T01:17:11.299257+0000","last_unstale":"2024-02-18T01:18:37.493811+0000","last_undegraded":"2024-02-18T01:18:37.493811+0000","last_fullsized":"2024-02-18T01:18:37.493811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"61'22","reported_seq":62,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.587036+0000","last_change":"2024-02-18T01:17:12.826047+0000","last_active":"2024-02-18T01:18:36.587036+0000","last_peered":"2024-02-18T01:18:36.587036+0000","last_clean":"2024-02-18T01:18:36.587036+0000","last_became_active":"2024-02-18T01:17:12.825489+0000","last_became_peered":"2024-02-18T01:17:12.825489+0000","last_unstale":"2024-02-18T01:18:36.587036+0000","last_undegraded":"2024-02-18T01:18:36.587036+0000","last_fullsized":"2024-02-18T01:18:36.587036+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":22,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":39,"num_read_kb":32,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546346+0000","last_change":"2024-02-18T01:17:16.836056+0000","last_active":"2024-02-18T01:18:36.546346+0000","last_peered":"2024-02-18T01:18:36.546346+0000","last_clean":"2024-02-18T01:18:36.546346+0000","last_became_active":"2024-02-18T01:17:16.835730+0000","last_became_peered":"2024-02-18T01:17:16.835730+0000","last_unstale":"2024-02-18T01:18:36.546346+0000","last_undegraded":"2024-02-18T01:18:36.546346+0000","last_fullsized":"2024-02-18T01:18:36.546346+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142752+0000","last_change":"2024-02-18T01:17:14.826870+0000","last_active":"2024-02-18T01:21:49.142752+0000","last_peered":"2024-02-18T01:21:49.142752+0000","last_clean":"2024-02-18T01:21:49.142752+0000","last_became_active":"2024-02-18T01:17:14.826541+0000","last_became_peered":"2024-02-18T01:17:14.826541+0000","last_unstale":"2024-02-18T01:21:49.142752+0000","last_undegraded":"2024-02-18T01:21:49.142752+0000","last_fullsized":"2024-02-18T01:21:49.142752+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548655+0000","last_change":"2024-02-18T01:17:10.710372+0000","last_active":"2024-02-18T01:18:36.548655+0000","last_peered":"2024-02-18T01:18:36.548655+0000","last_clean":"2024-02-18T01:18:36.548655+0000","last_became_active":"2024-02-18T01:17:10.709935+0000","last_became_peered":"2024-02-18T01:17:10.709935+0000","last_unstale":"2024-02-18T01:18:36.548655+0000","last_undegraded":"2024-02-18T01:18:36.548655+0000","last_fullsized":"2024-02-18T01:18:36.548655+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"61'15","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549975+0000","last_change":"2024-02-18T01:17:12.818041+0000","last_active":"2024-02-18T01:18:36.549975+0000","last_peered":"2024-02-18T01:18:36.549975+0000","last_clean":"2024-02-18T01:18:36.549975+0000","last_became_active":"2024-02-18T01:17:12.817638+0000","last_became_peered":"2024-02-18T01:17:12.817638+0000","last_unstale":"2024-02-18T01:18:36.549975+0000","last_undegraded":"2024-02-18T01:18:36.549975+0000","last_fullsized":"2024-02-18T01:18:36.549975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545935+0000","last_change":"2024-02-18T01:17:16.838573+0000","last_active":"2024-02-18T01:18:36.545935+0000","last_peered":"2024-02-18T01:18:36.545935+0000","last_clean":"2024-02-18T01:18:36.545935+0000","last_became_active":"2024-02-18T01:17:16.837423+0000","last_became_peered":"2024-02-18T01:17:16.837423+0000","last_unstale":"2024-02-18T01:18:36.545935+0000","last_undegraded":"2024-02-18T01:18:36.545935+0000","last_fullsized":"2024-02-18T01:18:36.545935+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586660+0000","last_change":"2024-02-18T01:17:14.827137+0000","last_active":"2024-02-18T01:18:36.586660+0000","last_peered":"2024-02-18T01:18:36.586660+0000","last_clean":"2024-02-18T01:18:36.586660+0000","last_became_active":"2024-02-18T01:17:14.826679+0000","last_became_peered":"2024-02-18T01:17:14.826679+0000","last_unstale":"2024-02-18T01:18:36.586660+0000","last_undegraded":"2024-02-18T01:18:36.586660+0000","last_fullsized":"2024-02-18T01:18:36.586660+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494245+0000","last_change":"2024-02-18T01:17:10.715366+0000","last_active":"2024-02-18T01:18:37.494245+0000","last_peered":"2024-02-18T01:18:37.494245+0000","last_clean":"2024-02-18T01:18:37.494245+0000","last_became_active":"2024-02-18T01:17:10.715068+0000","last_became_peered":"2024-02-18T01:17:10.715068+0000","last_unstale":"2024-02-18T01:18:37.494245+0000","last_undegraded":"2024-02-18T01:18:37.494245+0000","last_fullsized":"2024-02-18T01:18:37.494245+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"61'21","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551630+0000","last_change":"2024-02-18T01:17:12.824923+0000","last_active":"2024-02-18T01:18:36.551630+0000","last_peered":"2024-02-18T01:18:36.551630+0000","last_clean":"2024-02-18T01:18:36.551630+0000","last_became_active":"2024-02-18T01:17:12.824481+0000","last_became_peered":"2024-02-18T01:17:12.824481+0000","last_unstale":"2024-02-18T01:18:36.551630+0000","last_undegraded":"2024-02-18T01:18:36.551630+0000","last_fullsized":"2024-02-18T01:18:36.551630+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":21,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494298+0000","last_change":"2024-02-18T01:17:17.092997+0000","last_active":"2024-02-18T01:18:37.494298+0000","last_peered":"2024-02-18T01:18:37.494298+0000","last_clean":"2024-02-18T01:18:37.494298+0000","last_became_active":"2024-02-18T01:17:17.092646+0000","last_became_peered":"2024-02-18T01:17:17.092646+0000","last_unstale":"2024-02-18T01:18:37.494298+0000","last_undegraded":"2024-02-18T01:18:37.494298+0000","last_fullsized":"2024-02-18T01:18:37.494298+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549658+0000","last_change":"2024-02-18T01:17:14.838616+0000","last_active":"2024-02-18T01:18:36.549658+0000","last_peered":"2024-02-18T01:18:36.549658+0000","last_clean":"2024-02-18T01:18:36.549658+0000","last_became_active":"2024-02-18T01:17:14.838269+0000","last_became_peered":"2024-02-18T01:17:14.838269+0000","last_unstale":"2024-02-18T01:18:36.549658+0000","last_undegraded":"2024-02-18T01:18:36.549658+0000","last_fullsized":"2024-02-18T01:18:36.549658+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547590+0000","last_change":"2024-02-18T01:17:10.708083+0000","last_active":"2024-02-18T01:18:36.547590+0000","last_peered":"2024-02-18T01:18:36.547590+0000","last_clean":"2024-02-18T01:18:36.547590+0000","last_became_active":"2024-02-18T01:17:10.707586+0000","last_became_peered":"2024-02-18T01:17:10.707586+0000","last_unstale":"2024-02-18T01:18:36.547590+0000","last_undegraded":"2024-02-18T01:18:36.547590+0000","last_fullsized":"2024-02-18T01:18:36.547590+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"61'12","reported_seq":41,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586556+0000","last_change":"2024-02-18T01:17:12.826487+0000","last_active":"2024-02-18T01:18:36.586556+0000","last_peered":"2024-02-18T01:18:36.586556+0000","last_clean":"2024-02-18T01:18:36.586556+0000","last_became_active":"2024-02-18T01:17:12.825723+0000","last_became_peered":"2024-02-18T01:17:12.825723+0000","last_unstale":"2024-02-18T01:18:36.586556+0000","last_undegraded":"2024-02-18T01:18:36.586556+0000","last_fullsized":"2024-02-18T01:18:36.586556+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":21,"num_read_kb":17,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549393+0000","last_change":"2024-02-18T01:17:17.093651+0000","last_active":"2024-02-18T01:18:36.549393+0000","last_peered":"2024-02-18T01:18:36.549393+0000","last_clean":"2024-02-18T01:18:36.549393+0000","last_became_active":"2024-02-18T01:17:17.092523+0000","last_became_peered":"2024-02-18T01:17:17.092523+0000","last_unstale":"2024-02-18T01:18:36.549393+0000","last_undegraded":"2024-02-18T01:18:36.549393+0000","last_fullsized":"2024-02-18T01:18:36.549393+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"54'2","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494721+0000","last_change":"2024-02-18T01:17:10.716828+0000","last_active":"2024-02-18T01:18:37.494721+0000","last_peered":"2024-02-18T01:18:37.494721+0000","last_clean":"2024-02-18T01:18:37.494721+0000","last_became_active":"2024-02-18T01:17:10.716565+0000","last_became_peered":"2024-02-18T01:17:10.716565+0000","last_unstale":"2024-02-18T01:18:37.494721+0000","last_undegraded":"2024-02-18T01:18:37.494721+0000","last_fullsized":"2024-02-18T01:18:37.494721+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"61'27","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548459+0000","last_change":"2024-02-18T01:17:12.819458+0000","last_active":"2024-02-18T01:18:36.548459+0000","last_peered":"2024-02-18T01:18:36.548459+0000","last_clean":"2024-02-18T01:18:36.548459+0000","last_became_active":"2024-02-18T01:17:12.819110+0000","last_became_peered":"2024-02-18T01:17:12.819110+0000","last_unstale":"2024-02-18T01:18:36.548459+0000","last_undegraded":"2024-02-18T01:18:36.548459+0000","last_fullsized":"2024-02-18T01:18:36.548459+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":27,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":50,"num_read_kb":39,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142748+0000","last_change":"2024-02-18T01:17:14.835396+0000","last_active":"2024-02-18T01:21:49.142748+0000","last_peered":"2024-02-18T01:21:49.142748+0000","last_clean":"2024-02-18T01:21:49.142748+0000","last_became_active":"2024-02-18T01:17:14.834731+0000","last_became_peered":"2024-02-18T01:17:14.834731+0000","last_unstale":"2024-02-18T01:21:49.142748+0000","last_undegraded":"2024-02-18T01:21:49.142748+0000","last_fullsized":"2024-02-18T01:21:49.142748+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585943+0000","last_change":"2024-02-18T01:17:16.842343+0000","last_active":"2024-02-18T01:18:36.585943+0000","last_peered":"2024-02-18T01:18:36.585943+0000","last_clean":"2024-02-18T01:18:36.585943+0000","last_became_active":"2024-02-18T01:17:16.841956+0000","last_became_peered":"2024-02-18T01:17:16.841956+0000","last_unstale":"2024-02-18T01:18:36.585943+0000","last_undegraded":"2024-02-18T01:18:36.585943+0000","last_fullsized":"2024-02-18T01:18:36.585943+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494632+0000","last_change":"2024-02-18T01:17:10.717021+0000","last_active":"2024-02-18T01:18:37.494632+0000","last_peered":"2024-02-18T01:18:37.494632+0000","last_clean":"2024-02-18T01:18:37.494632+0000","last_became_active":"2024-02-18T01:17:10.716745+0000","last_became_peered":"2024-02-18T01:17:10.716745+0000","last_unstale":"2024-02-18T01:18:37.494632+0000","last_undegraded":"2024-02-18T01:18:37.494632+0000","last_fullsized":"2024-02-18T01:18:37.494632+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"61'26","reported_seq":72,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545180+0000","last_change":"2024-02-18T01:17:12.829120+0000","last_active":"2024-02-18T01:18:36.545180+0000","last_peered":"2024-02-18T01:18:36.545180+0000","last_clean":"2024-02-18T01:18:36.545180+0000","last_became_active":"2024-02-18T01:17:12.828782+0000","last_became_peered":"2024-02-18T01:17:12.828782+0000","last_unstale":"2024-02-18T01:18:36.545180+0000","last_undegraded":"2024-02-18T01:18:36.545180+0000","last_fullsized":"2024-02-18T01:18:36.545180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":26,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":49,"num_read_kb":38,"num_write":26,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544942+0000","last_change":"2024-02-18T01:17:14.821976+0000","last_active":"2024-02-18T01:18:36.544942+0000","last_peered":"2024-02-18T01:18:36.544942+0000","last_clean":"2024-02-18T01:18:36.544942+0000","last_became_active":"2024-02-18T01:17:14.821625+0000","last_became_peered":"2024-02-18T01:17:14.821625+0000","last_unstale":"2024-02-18T01:18:36.544942+0000","last_undegraded":"2024-02-18T01:18:36.544942+0000","last_fullsized":"2024-02-18T01:18:36.544942+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.492999+0000","last_change":"2024-02-18T01:17:16.840263+0000","last_active":"2024-02-18T01:18:37.492999+0000","last_peered":"2024-02-18T01:18:37.492999+0000","last_clean":"2024-02-18T01:18:37.492999+0000","last_became_active":"2024-02-18T01:17:16.840012+0000","last_became_peered":"2024-02-18T01:17:16.840012+0000","last_unstale":"2024-02-18T01:18:37.492999+0000","last_undegraded":"2024-02-18T01:18:37.492999+0000","last_fullsized":"2024-02-18T01:18:37.492999+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545548+0000","last_change":"2024-02-18T01:17:14.837962+0000","last_active":"2024-02-18T01:18:36.545548+0000","last_peered":"2024-02-18T01:18:36.545548+0000","last_clean":"2024-02-18T01:18:36.545548+0000","last_became_active":"2024-02-18T01:17:14.837597+0000","last_became_peered":"2024-02-18T01:17:14.837597+0000","last_unstale":"2024-02-18T01:18:36.545548+0000","last_undegraded":"2024-02-18T01:18:36.545548+0000","last_fullsized":"2024-02-18T01:18:36.545548+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.495112+0000","last_change":"2024-02-18T01:17:10.713527+0000","last_active":"2024-02-18T01:18:37.495112+0000","last_peered":"2024-02-18T01:18:37.495112+0000","last_clean":"2024-02-18T01:18:37.495112+0000","last_became_active":"2024-02-18T01:17:10.713162+0000","last_became_peered":"2024-02-18T01:17:10.713162+0000","last_unstale":"2024-02-18T01:18:37.495112+0000","last_undegraded":"2024-02-18T01:18:37.495112+0000","last_fullsized":"2024-02-18T01:18:37.495112+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"61'25","reported_seq":63,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547975+0000","last_change":"2024-02-18T01:17:12.828783+0000","last_active":"2024-02-18T01:18:36.547975+0000","last_peered":"2024-02-18T01:18:36.547975+0000","last_clean":"2024-02-18T01:18:36.547975+0000","last_became_active":"2024-02-18T01:17:12.828435+0000","last_became_peered":"2024-02-18T01:17:12.828435+0000","last_unstale":"2024-02-18T01:18:36.547975+0000","last_undegraded":"2024-02-18T01:18:36.547975+0000","last_fullsized":"2024-02-18T01:18:36.547975+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":44,"num_read_kb":32,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549338+0000","last_change":"2024-02-18T01:17:16.835057+0000","last_active":"2024-02-18T01:18:36.549338+0000","last_peered":"2024-02-18T01:18:36.549338+0000","last_clean":"2024-02-18T01:18:36.549338+0000","last_became_active":"2024-02-18T01:17:16.834338+0000","last_became_peered":"2024-02-18T01:17:16.834338+0000","last_unstale":"2024-02-18T01:18:36.549338+0000","last_undegraded":"2024-02-18T01:18:36.549338+0000","last_fullsized":"2024-02-18T01:18:36.549338+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545095+0000","last_change":"2024-02-18T01:17:14.823641+0000","last_active":"2024-02-18T01:18:36.545095+0000","last_peered":"2024-02-18T01:18:36.545095+0000","last_clean":"2024-02-18T01:18:36.545095+0000","last_became_active":"2024-02-18T01:17:14.823276+0000","last_became_peered":"2024-02-18T01:17:14.823276+0000","last_unstale":"2024-02-18T01:18:36.545095+0000","last_undegraded":"2024-02-18T01:18:36.545095+0000","last_fullsized":"2024-02-18T01:18:36.545095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549160+0000","last_change":"2024-02-18T01:17:10.706508+0000","last_active":"2024-02-18T01:18:36.549160+0000","last_peered":"2024-02-18T01:18:36.549160+0000","last_clean":"2024-02-18T01:18:36.549160+0000","last_became_active":"2024-02-18T01:17:10.705707+0000","last_became_peered":"2024-02-18T01:17:10.705707+0000","last_unstale":"2024-02-18T01:18:36.549160+0000","last_undegraded":"2024-02-18T01:18:36.549160+0000","last_fullsized":"2024-02-18T01:18:36.549160+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"61'15","reported_seq":48,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550033+0000","last_change":"2024-02-18T01:17:12.819522+0000","last_active":"2024-02-18T01:18:36.550033+0000","last_peered":"2024-02-18T01:18:36.550033+0000","last_clean":"2024-02-18T01:18:36.550033+0000","last_became_active":"2024-02-18T01:17:12.819201+0000","last_became_peered":"2024-02-18T01:17:12.819201+0000","last_unstale":"2024-02-18T01:18:36.550033+0000","last_undegraded":"2024-02-18T01:18:36.550033+0000","last_fullsized":"2024-02-18T01:18:36.550033+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":22,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586671+0000","last_change":"2024-02-18T01:17:16.838837+0000","last_active":"2024-02-18T01:18:36.586671+0000","last_peered":"2024-02-18T01:18:36.586671+0000","last_clean":"2024-02-18T01:18:36.586671+0000","last_became_active":"2024-02-18T01:17:16.838546+0000","last_became_peered":"2024-02-18T01:17:16.838546+0000","last_unstale":"2024-02-18T01:18:36.586671+0000","last_undegraded":"2024-02-18T01:18:36.586671+0000","last_fullsized":"2024-02-18T01:18:36.586671+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548888+0000","last_change":"2024-02-18T01:17:09.696709+0000","last_active":"2024-02-18T01:18:36.548888+0000","last_peered":"2024-02-18T01:18:36.548888+0000","last_clean":"2024-02-18T01:18:36.548888+0000","last_became_active":"2024-02-18T01:17:09.696408+0000","last_became_peered":"2024-02-18T01:17:09.696408+0000","last_unstale":"2024-02-18T01:18:36.548888+0000","last_undegraded":"2024-02-18T01:18:36.548888+0000","last_fullsized":"2024-02-18T01:18:36.548888+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545015+0000","last_change":"2024-02-18T01:17:10.703719+0000","last_active":"2024-02-18T01:18:36.545015+0000","last_peered":"2024-02-18T01:18:36.545015+0000","last_clean":"2024-02-18T01:18:36.545015+0000","last_became_active":"2024-02-18T01:17:10.703336+0000","last_became_peered":"2024-02-18T01:17:10.703336+0000","last_unstale":"2024-02-18T01:18:36.545015+0000","last_undegraded":"2024-02-18T01:18:36.545015+0000","last_fullsized":"2024-02-18T01:18:36.545015+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"61'34","reported_seq":136,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142756+0000","last_change":"2024-02-18T01:17:12.822478+0000","last_active":"2024-02-18T01:21:49.142756+0000","last_peered":"2024-02-18T01:21:49.142756+0000","last_clean":"2024-02-18T01:21:49.142756+0000","last_became_active":"2024-02-18T01:17:12.822148+0000","last_became_peered":"2024-02-18T01:17:12.822148+0000","last_unstale":"2024-02-18T01:21:49.142756+0000","last_undegraded":"2024-02-18T01:21:49.142756+0000","last_fullsized":"2024-02-18T01:21:49.142756+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":34,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":33,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":60,"num_read_kb":44,"num_write":32,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550355+0000","last_change":"2024-02-18T01:17:14.832997+0000","last_active":"2024-02-18T01:18:36.550355+0000","last_peered":"2024-02-18T01:18:36.550355+0000","last_clean":"2024-02-18T01:18:36.550355+0000","last_became_active":"2024-02-18T01:17:14.829833+0000","last_became_peered":"2024-02-18T01:17:14.829833+0000","last_unstale":"2024-02-18T01:18:36.550355+0000","last_undegraded":"2024-02-18T01:18:36.550355+0000","last_fullsized":"2024-02-18T01:18:36.550355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493228+0000","last_change":"2024-02-18T01:17:17.096876+0000","last_active":"2024-02-18T01:18:37.493228+0000","last_peered":"2024-02-18T01:18:37.493228+0000","last_clean":"2024-02-18T01:18:37.493228+0000","last_became_active":"2024-02-18T01:17:17.095788+0000","last_became_peered":"2024-02-18T01:17:17.095788+0000","last_unstale":"2024-02-18T01:18:37.493228+0000","last_undegraded":"2024-02-18T01:18:37.493228+0000","last_fullsized":"2024-02-18T01:18:37.493228+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543788+0000","last_change":"2024-02-18T01:17:11.299544+0000","last_active":"2024-02-18T01:18:36.543788+0000","last_peered":"2024-02-18T01:18:36.543788+0000","last_clean":"2024-02-18T01:18:36.543788+0000","last_became_active":"2024-02-18T01:17:11.298038+0000","last_became_peered":"2024-02-18T01:17:11.298038+0000","last_unstale":"2024-02-18T01:18:36.543788+0000","last_undegraded":"2024-02-18T01:18:36.543788+0000","last_fullsized":"2024-02-18T01:18:36.543788+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":20,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545286+0000","last_change":"2024-02-18T01:17:09.697042+0000","last_active":"2024-02-18T01:18:36.545286+0000","last_peered":"2024-02-18T01:18:36.545286+0000","last_clean":"2024-02-18T01:18:36.545286+0000","last_became_active":"2024-02-18T01:17:09.696626+0000","last_became_peered":"2024-02-18T01:17:09.696626+0000","last_unstale":"2024-02-18T01:18:36.545286+0000","last_undegraded":"2024-02-18T01:18:36.545286+0000","last_fullsized":"2024-02-18T01:18:36.545286+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"61'19","reported_seq":59,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548034+0000","last_change":"2024-02-18T01:17:12.814141+0000","last_active":"2024-02-18T01:18:36.548034+0000","last_peered":"2024-02-18T01:18:36.548034+0000","last_clean":"2024-02-18T01:18:36.548034+0000","last_became_active":"2024-02-18T01:17:12.813750+0000","last_became_peered":"2024-02-18T01:17:12.813750+0000","last_unstale":"2024-02-18T01:18:36.548034+0000","last_undegraded":"2024-02-18T01:18:36.548034+0000","last_fullsized":"2024-02-18T01:18:36.548034+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":19,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":42,"num_read_kb":30,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546051+0000","last_change":"2024-02-18T01:17:14.827887+0000","last_active":"2024-02-18T01:18:36.546051+0000","last_peered":"2024-02-18T01:18:36.546051+0000","last_clean":"2024-02-18T01:18:36.546051+0000","last_became_active":"2024-02-18T01:17:14.827544+0000","last_became_peered":"2024-02-18T01:17:14.827544+0000","last_unstale":"2024-02-18T01:18:36.546051+0000","last_undegraded":"2024-02-18T01:18:36.546051+0000","last_fullsized":"2024-02-18T01:18:36.546051+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"61'1","reported_seq":14,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550178+0000","last_change":"2024-02-18T01:17:16.839589+0000","last_active":"2024-02-18T01:18:36.550178+0000","last_peered":"2024-02-18T01:18:36.550178+0000","last_clean":"2024-02-18T01:18:36.550178+0000","last_became_active":"2024-02-18T01:17:16.839273+0000","last_became_peered":"2024-02-18T01:17:16.839273+0000","last_unstale":"2024-02-18T01:18:36.550178+0000","last_undegraded":"2024-02-18T01:18:36.550178+0000","last_fullsized":"2024-02-18T01:18:36.550178+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":23,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494049+0000","last_change":"2024-02-18T01:16:57.389903+0000","last_active":"2024-02-18T01:18:37.494049+0000","last_peered":"2024-02-18T01:18:37.494049+0000","last_clean":"2024-02-18T01:18:37.494049+0000","last_became_active":"2024-02-18T01:16:57.388815+0000","last_became_peered":"2024-02-18T01:16:57.388815+0000","last_unstale":"2024-02-18T01:18:37.494049+0000","last_undegraded":"2024-02-18T01:18:37.494049+0000","last_fullsized":"2024-02-18T01:18:37.494049+0000","mapping_epoch":50,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":51,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:13:29.577848+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:13:29.577848+0000","last_clean_scrub_stamp":"2024-02-18T01:13:29.577848+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586129+0000","last_change":"2024-02-18T01:17:10.709874+0000","last_active":"2024-02-18T01:18:36.586129+0000","last_peered":"2024-02-18T01:18:36.586129+0000","last_clean":"2024-02-18T01:18:36.586129+0000","last_became_active":"2024-02-18T01:17:10.709527+0000","last_became_peered":"2024-02-18T01:17:10.709527+0000","last_unstale":"2024-02-18T01:18:36.586129+0000","last_undegraded":"2024-02-18T01:18:36.586129+0000","last_fullsized":"2024-02-18T01:18:36.586129+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"61'14","reported_seq":39,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548924+0000","last_change":"2024-02-18T01:17:12.818782+0000","last_active":"2024-02-18T01:18:36.548924+0000","last_peered":"2024-02-18T01:18:36.548924+0000","last_clean":"2024-02-18T01:18:36.548924+0000","last_became_active":"2024-02-18T01:17:12.818420+0000","last_became_peered":"2024-02-18T01:17:12.818420+0000","last_unstale":"2024-02-18T01:18:36.548924+0000","last_undegraded":"2024-02-18T01:18:36.548924+0000","last_fullsized":"2024-02-18T01:18:36.548924+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":14,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.494932+0000","last_change":"2024-02-18T01:17:14.834512+0000","last_active":"2024-02-18T01:18:37.494932+0000","last_peered":"2024-02-18T01:18:37.494932+0000","last_clean":"2024-02-18T01:18:37.494932+0000","last_became_active":"2024-02-18T01:17:14.834201+0000","last_became_peered":"2024-02-18T01:17:14.834201+0000","last_unstale":"2024-02-18T01:18:37.494932+0000","last_undegraded":"2024-02-18T01:18:37.494932+0000","last_fullsized":"2024-02-18T01:18:37.494932+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544070+0000","last_change":"2024-02-18T01:17:16.840858+0000","last_active":"2024-02-18T01:18:36.544070+0000","last_peered":"2024-02-18T01:18:36.544070+0000","last_clean":"2024-02-18T01:18:36.544070+0000","last_became_active":"2024-02-18T01:17:16.840569+0000","last_became_peered":"2024-02-18T01:17:16.840569+0000","last_unstale":"2024-02-18T01:18:36.544070+0000","last_undegraded":"2024-02-18T01:18:36.544070+0000","last_fullsized":"2024-02-18T01:18:36.544070+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":26,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549613+0000","last_change":"2024-02-18T01:17:10.715267+0000","last_active":"2024-02-18T01:18:36.549613+0000","last_peered":"2024-02-18T01:18:36.549613+0000","last_clean":"2024-02-18T01:18:36.549613+0000","last_became_active":"2024-02-18T01:17:10.714928+0000","last_became_peered":"2024-02-18T01:17:10.714928+0000","last_unstale":"2024-02-18T01:18:36.549613+0000","last_undegraded":"2024-02-18T01:18:36.549613+0000","last_fullsized":"2024-02-18T01:18:36.549613+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":873,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'5","reported_seq":302,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:55.531341+0000","last_change":"2024-02-18T01:17:09.701392+0000","last_active":"2024-02-18T01:21:55.531341+0000","last_peered":"2024-02-18T01:21:55.531341+0000","last_clean":"2024-02-18T01:21:55.531341+0000","last_became_active":"2024-02-18T01:17:09.701105+0000","last_became_peered":"2024-02-18T01:17:09.701105+0000","last_unstale":"2024-02-18T01:21:55.531341+0000","last_undegraded":"2024-02-18T01:21:55.531341+0000","last_fullsized":"2024-02-18T01:21:55.531341+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:08.679234+0000","last_clean_scrub_stamp":"2024-02-18T01:17:08.679234+0000","log_size":5,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":279,"num_read_kb":274,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"61'12","reported_seq":58,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:38.602149+0000","last_change":"2024-02-18T01:18:38.602149+0000","last_active":"2024-02-18T01:18:38.602149+0000","last_peered":"2024-02-18T01:18:38.602149+0000","last_clean":"2024-02-18T01:18:38.602149+0000","last_became_active":"2024-02-18T01:18:37.562746+0000","last_became_peered":"2024-02-18T01:18:37.562746+0000","last_unstale":"2024-02-18T01:18:38.602149+0000","last_undegraded":"2024-02-18T01:18:38.602149+0000","last_fullsized":"2024-02-18T01:18:38.602149+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":12,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550346+0000","last_change":"2024-02-18T01:17:14.834128+0000","last_active":"2024-02-18T01:18:36.550346+0000","last_peered":"2024-02-18T01:18:36.550346+0000","last_clean":"2024-02-18T01:18:36.550346+0000","last_became_active":"2024-02-18T01:17:14.833788+0000","last_became_peered":"2024-02-18T01:17:14.833788+0000","last_unstale":"2024-02-18T01:18:36.550346+0000","last_undegraded":"2024-02-18T01:18:36.550346+0000","last_fullsized":"2024-02-18T01:18:36.550346+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.543623+0000","last_change":"2024-02-18T01:17:17.093645+0000","last_active":"2024-02-18T01:18:36.543623+0000","last_peered":"2024-02-18T01:18:36.543623+0000","last_clean":"2024-02-18T01:18:36.543623+0000","last_became_active":"2024-02-18T01:17:17.091841+0000","last_became_peered":"2024-02-18T01:17:17.091841+0000","last_unstale":"2024-02-18T01:18:36.543623+0000","last_undegraded":"2024-02-18T01:18:36.543623+0000","last_fullsized":"2024-02-18T01:18:36.543623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544240+0000","last_change":"2024-02-18T01:17:11.301831+0000","last_active":"2024-02-18T01:18:36.544240+0000","last_peered":"2024-02-18T01:18:36.544240+0000","last_clean":"2024-02-18T01:18:36.544240+0000","last_became_active":"2024-02-18T01:17:11.301510+0000","last_became_peered":"2024-02-18T01:17:11.301510+0000","last_unstale":"2024-02-18T01:18:36.544240+0000","last_undegraded":"2024-02-18T01:18:36.544240+0000","last_fullsized":"2024-02-18T01:18:36.544240+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"61'25","reported_seq":68,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549889+0000","last_change":"2024-02-18T01:17:12.820612+0000","last_active":"2024-02-18T01:18:36.549889+0000","last_peered":"2024-02-18T01:18:36.549889+0000","last_clean":"2024-02-18T01:18:36.549889+0000","last_became_active":"2024-02-18T01:17:12.819570+0000","last_became_peered":"2024-02-18T01:17:12.819570+0000","last_unstale":"2024-02-18T01:18:36.549889+0000","last_undegraded":"2024-02-18T01:18:36.549889+0000","last_fullsized":"2024-02-18T01:18:36.549889+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":25,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548166+0000","last_change":"2024-02-18T01:17:14.839830+0000","last_active":"2024-02-18T01:18:36.548166+0000","last_peered":"2024-02-18T01:18:36.548166+0000","last_clean":"2024-02-18T01:18:36.548166+0000","last_became_active":"2024-02-18T01:17:14.839502+0000","last_became_peered":"2024-02-18T01:17:14.839502+0000","last_unstale":"2024-02-18T01:18:36.548166+0000","last_undegraded":"2024-02-18T01:18:36.548166+0000","last_fullsized":"2024-02-18T01:18:36.548166+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548725+0000","last_change":"2024-02-18T01:17:16.838705+0000","last_active":"2024-02-18T01:18:36.548725+0000","last_peered":"2024-02-18T01:18:36.548725+0000","last_clean":"2024-02-18T01:18:36.548725+0000","last_became_active":"2024-02-18T01:17:16.838355+0000","last_became_peered":"2024-02-18T01:17:16.838355+0000","last_unstale":"2024-02-18T01:18:36.548725+0000","last_undegraded":"2024-02-18T01:18:36.548725+0000","last_fullsized":"2024-02-18T01:18:36.548725+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549894+0000","last_change":"2024-02-18T01:17:11.299454+0000","last_active":"2024-02-18T01:18:36.549894+0000","last_peered":"2024-02-18T01:18:36.549894+0000","last_clean":"2024-02-18T01:18:36.549894+0000","last_became_active":"2024-02-18T01:17:11.298786+0000","last_became_peered":"2024-02-18T01:17:11.298786+0000","last_unstale":"2024-02-18T01:18:36.549894+0000","last_undegraded":"2024-02-18T01:18:36.549894+0000","last_fullsized":"2024-02-18T01:18:36.549894+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"61'16","reported_seq":24,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:18:38.594750+0000","last_change":"2024-02-18T01:18:38.594750+0000","last_active":"2024-02-18T01:18:38.594750+0000","last_peered":"2024-02-18T01:18:38.594750+0000","last_clean":"2024-02-18T01:18:38.594750+0000","last_became_active":"2024-02-18T01:18:37.561512+0000","last_became_peered":"2024-02-18T01:18:37.561512+0000","last_unstale":"2024-02-18T01:18:38.594750+0000","last_undegraded":"2024-02-18T01:18:38.594750+0000","last_fullsized":"2024-02-18T01:18:38.594750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":16,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":21,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142374+0000","last_change":"2024-02-18T01:17:14.833420+0000","last_active":"2024-02-18T01:21:49.142374+0000","last_peered":"2024-02-18T01:21:49.142374+0000","last_clean":"2024-02-18T01:21:49.142374+0000","last_became_active":"2024-02-18T01:17:14.832798+0000","last_became_peered":"2024-02-18T01:17:14.832798+0000","last_unstale":"2024-02-18T01:21:49.142374+0000","last_undegraded":"2024-02-18T01:21:49.142374+0000","last_fullsized":"2024-02-18T01:21:49.142374+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:37.493124+0000","last_change":"2024-02-18T01:17:17.096220+0000","last_active":"2024-02-18T01:18:37.493124+0000","last_peered":"2024-02-18T01:18:37.493124+0000","last_clean":"2024-02-18T01:18:37.493124+0000","last_became_active":"2024-02-18T01:17:17.095785+0000","last_became_peered":"2024-02-18T01:17:17.095785+0000","last_unstale":"2024-02-18T01:18:37.493124+0000","last_undegraded":"2024-02-18T01:18:37.493124+0000","last_fullsized":"2024-02-18T01:18:37.493124+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.551518+0000","last_change":"2024-02-18T01:17:10.704177+0000","last_active":"2024-02-18T01:18:36.551518+0000","last_peered":"2024-02-18T01:18:36.551518+0000","last_clean":"2024-02-18T01:18:36.551518+0000","last_became_active":"2024-02-18T01:17:10.703563+0000","last_became_peered":"2024-02-18T01:17:10.703563+0000","last_unstale":"2024-02-18T01:18:36.551518+0000","last_undegraded":"2024-02-18T01:18:36.551518+0000","last_fullsized":"2024-02-18T01:18:36.551518+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"61'20","reported_seq":53,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544958+0000","last_change":"2024-02-18T01:17:12.826509+0000","last_active":"2024-02-18T01:18:36.544958+0000","last_peered":"2024-02-18T01:18:36.544958+0000","last_clean":"2024-02-18T01:18:36.544958+0000","last_became_active":"2024-02-18T01:17:12.826187+0000","last_became_peered":"2024-02-18T01:17:12.826187+0000","last_unstale":"2024-02-18T01:18:36.544958+0000","last_undegraded":"2024-02-18T01:18:36.544958+0000","last_fullsized":"2024-02-18T01:18:36.544958+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":20,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":33,"num_read_kb":25,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.0","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.141982+0000","last_change":"2024-02-18T01:17:14.824677+0000","last_active":"2024-02-18T01:21:49.141982+0000","last_peered":"2024-02-18T01:21:49.141982+0000","last_clean":"2024-02-18T01:21:49.141982+0000","last_became_active":"2024-02-18T01:17:14.824307+0000","last_became_peered":"2024-02-18T01:17:14.824307+0000","last_unstale":"2024-02-18T01:21:49.141982+0000","last_undegraded":"2024-02-18T01:21:49.141982+0000","last_fullsized":"2024-02-18T01:21:49.141982+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545702+0000","last_change":"2024-02-18T01:17:16.836790+0000","last_active":"2024-02-18T01:18:36.545702+0000","last_peered":"2024-02-18T01:18:36.545702+0000","last_clean":"2024-02-18T01:18:36.545702+0000","last_became_active":"2024-02-18T01:17:16.836495+0000","last_became_peered":"2024-02-18T01:17:16.836495+0000","last_unstale":"2024-02-18T01:18:36.545702+0000","last_undegraded":"2024-02-18T01:18:36.545702+0000","last_fullsized":"2024-02-18T01:18:36.545702+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586501+0000","last_change":"2024-02-18T01:17:10.709186+0000","last_active":"2024-02-18T01:18:36.586501+0000","last_peered":"2024-02-18T01:18:36.586501+0000","last_clean":"2024-02-18T01:18:36.586501+0000","last_became_active":"2024-02-18T01:17:10.708852+0000","last_became_peered":"2024-02-18T01:17:10.708852+0000","last_unstale":"2024-02-18T01:18:36.586501+0000","last_undegraded":"2024-02-18T01:18:36.586501+0000","last_fullsized":"2024-02-18T01:18:36.586501+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"61'30","reported_seq":79,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585520+0000","last_change":"2024-02-18T01:17:12.825039+0000","last_active":"2024-02-18T01:18:36.585520+0000","last_peered":"2024-02-18T01:18:36.585520+0000","last_clean":"2024-02-18T01:18:36.585520+0000","last_became_active":"2024-02-18T01:17:12.824666+0000","last_became_peered":"2024-02-18T01:17:12.824666+0000","last_unstale":"2024-02-18T01:18:36.585520+0000","last_undegraded":"2024-02-18T01:18:36.585520+0000","last_fullsized":"2024-02-18T01:18:36.585520+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":30,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":220,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":58,"num_read_kb":44,"num_write":32,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544713+0000","last_change":"2024-02-18T01:17:14.824194+0000","last_active":"2024-02-18T01:18:36.544713+0000","last_peered":"2024-02-18T01:18:36.544713+0000","last_clean":"2024-02-18T01:18:36.544713+0000","last_became_active":"2024-02-18T01:17:14.823349+0000","last_became_peered":"2024-02-18T01:17:14.823349+0000","last_unstale":"2024-02-18T01:18:36.544713+0000","last_undegraded":"2024-02-18T01:18:36.544713+0000","last_fullsized":"2024-02-18T01:18:36.544713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.549693+0000","last_change":"2024-02-18T01:17:16.837880+0000","last_active":"2024-02-18T01:18:36.549693+0000","last_peered":"2024-02-18T01:18:36.549693+0000","last_clean":"2024-02-18T01:18:36.549693+0000","last_became_active":"2024-02-18T01:17:16.837631+0000","last_became_peered":"2024-02-18T01:17:16.837631+0000","last_unstale":"2024-02-18T01:18:36.549693+0000","last_undegraded":"2024-02-18T01:18:36.549693+0000","last_fullsized":"2024-02-18T01:18:36.549693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.142531+0000","last_change":"2024-02-18T01:17:14.832992+0000","last_active":"2024-02-18T01:21:49.142531+0000","last_peered":"2024-02-18T01:21:49.142531+0000","last_clean":"2024-02-18T01:21:49.142531+0000","last_became_active":"2024-02-18T01:17:14.832627+0000","last_became_peered":"2024-02-18T01:17:14.832627+0000","last_unstale":"2024-02-18T01:21:49.142531+0000","last_undegraded":"2024-02-18T01:21:49.142531+0000","last_fullsized":"2024-02-18T01:21:49.142531+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.585546+0000","last_change":"2024-02-18T01:17:10.710402+0000","last_active":"2024-02-18T01:18:36.585546+0000","last_peered":"2024-02-18T01:18:36.585546+0000","last_clean":"2024-02-18T01:18:36.585546+0000","last_became_active":"2024-02-18T01:17:10.710096+0000","last_became_peered":"2024-02-18T01:17:10.710096+0000","last_unstale":"2024-02-18T01:18:36.585546+0000","last_undegraded":"2024-02-18T01:18:36.585546+0000","last_fullsized":"2024-02-18T01:18:36.585546+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"61'15","reported_seq":42,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548084+0000","last_change":"2024-02-18T01:17:12.825237+0000","last_active":"2024-02-18T01:18:36.548084+0000","last_peered":"2024-02-18T01:18:36.548084+0000","last_clean":"2024-02-18T01:18:36.548084+0000","last_became_active":"2024-02-18T01:17:12.824964+0000","last_became_peered":"2024-02-18T01:17:12.824964+0000","last_unstale":"2024-02-18T01:18:36.548084+0000","last_undegraded":"2024-02-18T01:18:36.548084+0000","last_fullsized":"2024-02-18T01:18:36.548084+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":15,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586435+0000","last_change":"2024-02-18T01:17:17.095044+0000","last_active":"2024-02-18T01:18:36.586435+0000","last_peered":"2024-02-18T01:18:36.586435+0000","last_clean":"2024-02-18T01:18:36.586435+0000","last_became_active":"2024-02-18T01:17:17.094630+0000","last_became_peered":"2024-02-18T01:17:17.094630+0000","last_unstale":"2024-02-18T01:18:36.586435+0000","last_undegraded":"2024-02-18T01:18:36.586435+0000","last_fullsized":"2024-02-18T01:18:36.586435+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.550286+0000","last_change":"2024-02-18T01:17:14.837188+0000","last_active":"2024-02-18T01:18:36.550286+0000","last_peered":"2024-02-18T01:18:36.550286+0000","last_clean":"2024-02-18T01:18:36.550286+0000","last_became_active":"2024-02-18T01:17:14.836859+0000","last_became_peered":"2024-02-18T01:17:14.836859+0000","last_unstale":"2024-02-18T01:18:36.550286+0000","last_undegraded":"2024-02-18T01:18:36.550286+0000","last_fullsized":"2024-02-18T01:18:36.550286+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544750+0000","last_change":"2024-02-18T01:17:11.298329+0000","last_active":"2024-02-18T01:18:36.544750+0000","last_peered":"2024-02-18T01:18:36.544750+0000","last_clean":"2024-02-18T01:18:36.544750+0000","last_became_active":"2024-02-18T01:17:11.297607+0000","last_became_peered":"2024-02-18T01:17:11.297607+0000","last_unstale":"2024-02-18T01:18:36.544750+0000","last_undegraded":"2024-02-18T01:18:36.544750+0000","last_fullsized":"2024-02-18T01:18:36.544750+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"61'17","reported_seq":51,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546296+0000","last_change":"2024-02-18T01:17:12.826584+0000","last_active":"2024-02-18T01:18:36.546296+0000","last_peered":"2024-02-18T01:18:36.546296+0000","last_clean":"2024-02-18T01:18:36.546296+0000","last_became_active":"2024-02-18T01:17:12.826226+0000","last_became_peered":"2024-02-18T01:17:12.826226+0000","last_unstale":"2024-02-18T01:18:36.546296+0000","last_undegraded":"2024-02-18T01:18:36.546296+0000","last_fullsized":"2024-02-18T01:18:36.546296+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":32,"num_read_kb":24,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545858+0000","last_change":"2024-02-18T01:17:16.836838+0000","last_active":"2024-02-18T01:18:36.545858+0000","last_peered":"2024-02-18T01:18:36.545858+0000","last_clean":"2024-02-18T01:18:36.545858+0000","last_became_active":"2024-02-18T01:17:16.836509+0000","last_became_peered":"2024-02-18T01:17:16.836509+0000","last_unstale":"2024-02-18T01:18:36.545858+0000","last_undegraded":"2024-02-18T01:18:36.545858+0000","last_fullsized":"2024-02-18T01:18:36.545858+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":15,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.544527+0000","last_change":"2024-02-18T01:17:14.828948+0000","last_active":"2024-02-18T01:18:36.544527+0000","last_peered":"2024-02-18T01:18:36.544527+0000","last_clean":"2024-02-18T01:18:36.544527+0000","last_became_active":"2024-02-18T01:17:14.828368+0000","last_became_peered":"2024-02-18T01:17:14.828368+0000","last_unstale":"2024-02-18T01:18:36.544527+0000","last_undegraded":"2024-02-18T01:18:36.544527+0000","last_fullsized":"2024-02-18T01:18:36.544527+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"54'1","reported_seq":25,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.548265+0000","last_change":"2024-02-18T01:17:10.707232+0000","last_active":"2024-02-18T01:18:36.548265+0000","last_peered":"2024-02-18T01:18:36.548265+0000","last_clean":"2024-02-18T01:18:36.548265+0000","last_became_active":"2024-02-18T01:17:10.706178+0000","last_became_peered":"2024-02-18T01:17:10.706178+0000","last_unstale":"2024-02-18T01:18:36.548265+0000","last_undegraded":"2024-02-18T01:18:36.548265+0000","last_fullsized":"2024-02-18T01:18:36.548265+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":358,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"61'17","reported_seq":45,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.546247+0000","last_change":"2024-02-18T01:17:12.822070+0000","last_active":"2024-02-18T01:18:36.546247+0000","last_peered":"2024-02-18T01:18:36.546247+0000","last_clean":"2024-02-18T01:18:36.546247+0000","last_became_active":"2024-02-18T01:17:12.821732+0000","last_became_peered":"2024-02-18T01:17:12.821732+0000","last_unstale":"2024-02-18T01:18:36.546247+0000","last_undegraded":"2024-02-18T01:18:36.546247+0000","last_fullsized":"2024-02-18T01:18:36.546247+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":17,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":13,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.547064+0000","last_change":"2024-02-18T01:17:16.837596+0000","last_active":"2024-02-18T01:18:36.547064+0000","last_peered":"2024-02-18T01:18:36.547064+0000","last_clean":"2024-02-18T01:18:36.547064+0000","last_became_active":"2024-02-18T01:17:16.836366+0000","last_became_peered":"2024-02-18T01:17:16.836366+0000","last_unstale":"2024-02-18T01:18:36.547064+0000","last_undegraded":"2024-02-18T01:18:36.547064+0000","last_fullsized":"2024-02-18T01:18:36.547064+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:15.814851+0000","last_clean_scrub_stamp":"2024-02-18T01:17:15.814851+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"61'8","reported_seq":79,"reported_epoch":64,"state":"active+clean","last_fresh":"2024-02-18T01:21:49.141986+0000","last_change":"2024-02-18T01:17:14.829883+0000","last_active":"2024-02-18T01:21:49.141986+0000","last_peered":"2024-02-18T01:21:49.141986+0000","last_clean":"2024-02-18T01:21:49.141986+0000","last_became_active":"2024-02-18T01:17:14.828822+0000","last_became_peered":"2024-02-18T01:17:14.828822+0000","last_unstale":"2024-02-18T01:21:49.141986+0000","last_undegraded":"2024-02-18T01:21:49.141986+0000","last_fullsized":"2024-02-18T01:21:49.141986+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:13.802491+0000","last_clean_scrub_stamp":"2024-02-18T01:17:13.802491+0000","log_size":8,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":19,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.586170+0000","last_change":"2024-02-18T01:17:10.704860+0000","last_active":"2024-02-18T01:18:36.586170+0000","last_peered":"2024-02-18T01:18:36.586170+0000","last_clean":"2024-02-18T01:18:36.586170+0000","last_became_active":"2024-02-18T01:17:10.704554+0000","last_became_peered":"2024-02-18T01:17:10.704554+0000","last_unstale":"2024-02-18T01:18:36.586170+0000","last_undegraded":"2024-02-18T01:18:36.586170+0000","last_fullsized":"2024-02-18T01:18:36.586170+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:09.681656+0000","last_clean_scrub_stamp":"2024-02-18T01:17:09.681656+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"61'10","reported_seq":32,"reported_epoch":63,"state":"active+clean","last_fresh":"2024-02-18T01:18:36.545605+0000","last_change":"2024-02-18T01:17:12.825708+0000","last_active":"2024-02-18T01:18:36.545605+0000","last_peered":"2024-02-18T01:18:36.545605+0000","last_clean":"2024-02-18T01:18:36.545605+0000","last_became_active":"2024-02-18T01:17:12.825434+0000","last_became_peered":"2024-02-18T01:17:12.825434+0000","last_unstale":"2024-02-18T01:18:36.545605+0000","last_undegraded":"2024-02-18T01:18:36.545605+0000","last_fullsized":"2024-02-18T01:18:36.545605+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-02-18T01:17:11.796377+0000","last_clean_scrub_stamp":"2024-02-18T01:17:11.796377+0000","log_size":10,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":209,"num_object_clones":0,"num_object_copies":627,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":209,"num_whiteouts":0,"num_read":1082,"num_read_kb":807,"num_write":609,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":17,"num_bytes_recovered":110,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":585,"ondisk_log_size":585,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":1323,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":22,"num_read_kb":22,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":3,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":279,"num_read_kb":274,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":9,"acting":9,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":53,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297724,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447428,"statfs":{"total":95995035648,"available":95690166272,"internally_reserved":0,"allocated":643072,"data_stored":244356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.86699999999999999,"15min":0.80500000000000005},"min":{"1min":0.61599999999999999,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.878,"15min":0.84199999999999997},"min":{"1min":0.66600000000000004,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.98899999999999999}]},{"osd":1,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.89600000000000002,"15min":0.82999999999999996},"min":{"1min":0.57299999999999995,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0720000000000001,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91800000000000004,"15min":0.89900000000000002},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3740000000000001,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.2030000000000001}]},{"osd":2,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.90500000000000003,"5min":0.872,"15min":0.80700000000000005},"min":{"1min":0.63,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.081,"5min":1.266,"15min":1.266},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.94899999999999995,"5min":0.91600000000000004,"15min":0.86199999999999999},"min":{"1min":0.64000000000000001,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.2929999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":1.226}]},{"osd":3,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":0.92900000000000005,"15min":0.88900000000000001},"min":{"1min":0.70999999999999996,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.2250000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":1.0209999999999999,"5min":1.01,"15min":0.995},"min":{"1min":0.80700000000000005,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2310000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.80800000000000005}]},{"osd":4,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":0.84599999999999997,"15min":0.81499999999999995},"min":{"1min":0.65700000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.194,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.046,"5min":0.99199999999999999,"15min":0.94999999999999996},"min":{"1min":0.70799999999999996,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.2809999999999999,"5min":1.456,"15min":1.456},"last":0.94599999999999995}]},{"osd":5,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":0.94299999999999995,"15min":0.90500000000000003},"min":{"1min":0.66100000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.331,"5min":1.3819999999999999,"15min":1.3819999999999999},"last":1.093},{"interface":"front","average":{"1min":1,"5min":0.95199999999999996,"15min":0.92200000000000004},"min":{"1min":0.65200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.226,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.96899999999999997}]},{"osd":6,"last update":"Sun Feb 18 01:21:01 2024","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.048,"15min":1.04},"min":{"1min":0.76600000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.335,"5min":1.508,"15min":1.508},"last":1.2549999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0129999999999999,"15min":0.98699999999999999},"min":{"1min":0.81200000000000006,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.484,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":1.1639999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593799,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297708,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447444,"statfs":{"total":95995035648,"available":95690182656,"internally_reserved":0,"allocated":626688,"data_stored":243959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.88800000000000001,"15min":0.85499999999999998},"min":{"1min":0.65200000000000002,"5min":0.624,"15min":0.624},"max":{"1min":1.29,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.995},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.89800000000000002,"15min":0.93300000000000005},"min":{"1min":0.65700000000000003,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.3580000000000001,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.872}]},{"osd":1,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.89300000000000002,"15min":0.873},"min":{"1min":0.64400000000000002,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.113,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.879,"5min":0.86299999999999999,"15min":0.85499999999999998},"min":{"1min":0.61099999999999999,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":0.66100000000000003}]},{"osd":2,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.0309999999999999,"5min":0.97399999999999998,"15min":0.96199999999999997},"min":{"1min":0.85999999999999999,"5min":0.62,"15min":0.62},"max":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.995,"5min":0.997,"15min":0.98599999999999999},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2949999999999999,"5min":1.528,"15min":1.528},"last":0.70699999999999996}]},{"osd":3,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.97399999999999998,"15min":0.96799999999999997},"min":{"1min":0.71099999999999997,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.26,"5min":1.542,"15min":1.542},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.98199999999999998,"5min":0.95399999999999996,"15min":0.98099999999999998},"min":{"1min":0.75,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.89100000000000001}]},{"osd":4,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90800000000000003,"15min":0.93600000000000005},"min":{"1min":0.65000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.3400000000000001,"5min":1.738,"15min":1.738},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0449999999999999,"15min":1.0269999999999999},"min":{"1min":0.69099999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.4179999999999999,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1950000000000001}]},{"osd":5,"last update":"Sun Feb 18 01:21:23 2024","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":0.98599999999999999,"15min":1.008},"min":{"1min":0.68799999999999994,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.25,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.0289999999999999},{"interface":"front","average":{"1min":1.159,"5min":1.109,"15min":1.0820000000000001},"min":{"1min":0.90900000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.415,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":1.238}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.1220000000000001,"15min":1.111},"min":{"1min":0.80500000000000005,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4339999999999999,"5min":1.8939999999999999,"15min":1.8939999999999999},"last":1.113},{"interface":"front","average":{"1min":1.238,"5min":1.204,"15min":1.173},"min":{"1min":1.0449999999999999,"5min":0.63400000000000001,"15min":0.63400000000000001},"max":{"1min":1.464,"5min":1.742,"15min":1.742},"last":1.2829999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574960,"num_pgs":54,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297760,"kb_used_data":664,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447392,"statfs":{"total":95995035648,"available":95690129408,"internally_reserved":0,"allocated":679936,"data_stored":245542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:20:57 2024","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69799999999999995,"15min":0.57199999999999995},"min":{"1min":0.54200000000000004,"5min":0.41399999999999998,"15min":0.30499999999999999},"max":{"1min":1.0189999999999999,"5min":1.149,"15min":1.149},"last":1.7509999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.745,"15min":0.63},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.38500000000000001},"max":{"1min":0.95999999999999996,"5min":1.012,"15min":1.1359999999999999},"last":0.64200000000000002}]},{"osd":2,"last update":"Sun Feb 18 01:21:49 2024","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.77700000000000002,"15min":0.71099999999999997},"min":{"1min":0.48999999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.94299999999999995,"5min":1.113,"15min":1.113},"last":1.5069999999999999},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.755,"15min":0.70099999999999996},"min":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.41299999999999998},"max":{"1min":1.0760000000000001,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Sun Feb 18 01:21:24 2024","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.55100000000000005,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":1.1240000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.159},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.90100000000000002,"15min":0.84799999999999998},"min":{"1min":0.65200000000000002,"5min":0.373,"15min":0.373},"max":{"1min":1.331,"5min":2.504,"15min":2.504},"last":0.71199999999999997}]},{"osd":4,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":0.93300000000000005,"15min":0.91000000000000003},"min":{"1min":0.67300000000000004,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.2829999999999999,"5min":1.96,"15min":1.96},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.044,"15min":0.99399999999999999},"min":{"1min":0.76300000000000001,"5min":0.755,"15min":0.72599999999999998},"max":{"1min":1.393,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.0700000000000001}]},{"osd":5,"last update":"Sun Feb 18 01:21:49 2024","interfaces":[{"interface":"back","average":{"1min":1.121,"5min":1.1000000000000001,"15min":1.0580000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.4890000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.746},{"interface":"front","average":{"1min":1.097,"5min":1.0720000000000001,"15min":1.014},"min":{"1min":0.69899999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.569,"5min":1.855,"15min":1.855},"last":0.91200000000000003}]},{"osd":6,"last update":"Sun Feb 18 01:21:24 2024","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.099,"15min":1.091},"min":{"1min":0.78200000000000003,"5min":0.74299999999999999,"15min":0.74299999999999999},"max":{"1min":1.3660000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.1499999999999999,"15min":1.1319999999999999},"min":{"1min":0.877,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.4990000000000001,"5min":1.698,"15min":1.698},"last":0.95499999999999996}]},{"osd":7,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.1679999999999999,"15min":1.1599999999999999},"min":{"1min":0.90900000000000003,"5min":0.84499999999999997,"15min":0.84499999999999997},"max":{"1min":1.4390000000000001,"5min":2.1779999999999999,"15min":2.1779999999999999},"last":1.1319999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.212,"15min":1.1910000000000001},"min":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"max":{"1min":1.5589999999999999,"5min":1.6659999999999999,"15min":1.6659999999999999},"last":1.242}]}]},{"osd":0,"up_from":8,"seq":34359738488,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297732,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447420,"statfs":{"total":95995035648,"available":95690158080,"internally_reserved":0,"allocated":651264,"data_stored":245241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71599999999999997,"15min":0.57799999999999996},"min":{"1min":0.52800000000000002,"5min":0.40100000000000002,"15min":0.38},"max":{"1min":0.92100000000000004,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.74199999999999999,"15min":0.628},"min":{"1min":0.54200000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.66000000000000003}]},{"osd":2,"last update":"Sun Feb 18 01:21:52 2024","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79700000000000004,"15min":0.70299999999999996},"min":{"1min":0.48699999999999999,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.02,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.629},{"interface":"front","average":{"1min":0.76400000000000001,"5min":0.80400000000000005,"15min":0.73599999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.94899999999999995,"5min":1.149,"15min":1.149},"last":0.85099999999999998}]},{"osd":3,"last update":"Sun Feb 18 01:21:21 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88100000000000001,"15min":0.78700000000000003},"min":{"1min":0.68999999999999995,"5min":0.58599999999999997,"15min":0.51300000000000001},"max":{"1min":1.5189999999999999,"5min":1.5189999999999999,"15min":1.5189999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.89600000000000002,"5min":0.94499999999999995,"15min":0.85199999999999998},"min":{"1min":0.746,"5min":0.56000000000000005,"15min":0.55300000000000005},"max":{"1min":1.1659999999999999,"5min":1.3560000000000001,"15min":1.3560000000000001},"last":0.91900000000000004}]},{"osd":4,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.002,"15min":0.93500000000000005},"min":{"1min":0.73499999999999999,"5min":0.71599999999999997,"15min":0.68500000000000005},"max":{"1min":1.21,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.96499999999999997,"5min":0.92700000000000005,"15min":0.88500000000000001},"min":{"1min":0.75700000000000001,"5min":0.60199999999999998,"15min":0.60199999999999998},"max":{"1min":1.1279999999999999,"5min":1.284,"15min":1.284},"last":0.93999999999999995}]},{"osd":5,"last update":"Sun Feb 18 01:21:38 2024","interfaces":[{"interface":"back","average":{"1min":1.0109999999999999,"5min":0.96999999999999997,"15min":0.94599999999999995},"min":{"1min":0.73099999999999998,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.155,"15min":1.099},"min":{"1min":0.79500000000000004,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.169}]},{"osd":6,"last update":"Sun Feb 18 01:21:17 2024","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.1259999999999999,"15min":1.1060000000000001},"min":{"1min":0.90000000000000002,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":1.5549999999999999,"5min":1.5640000000000001,"15min":1.5640000000000001},"last":1.2130000000000001},{"interface":"front","average":{"1min":1.113,"5min":1.0580000000000001,"15min":1.0660000000000001},"min":{"1min":0.90600000000000003,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.403,"5min":1.444,"15min":1.444},"last":1.1879999999999999}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2110000000000001,"15min":1.1779999999999999},"min":{"1min":1.046,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.085},{"interface":"front","average":{"1min":1.218,"5min":1.23,"15min":1.2070000000000001},"min":{"1min":0.995,"5min":0.92000000000000004,"15min":0.92000000000000004},"max":{"1min":1.5169999999999999,"5min":1.675,"15min":1.675},"last":1.2849999999999999}]}]},{"osd":2,"up_from":18,"seq":77309411432,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297708,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447444,"statfs":{"total":95995035648,"available":95690182656,"internally_reserved":0,"allocated":626688,"data_stored":243844,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.70599999999999996,"15min":0.59199999999999997},"min":{"1min":0.56399999999999995,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.4119999999999999,"5min":1.4119999999999999,"15min":1.4119999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.753,"5min":0.72299999999999998,"15min":0.63900000000000001},"min":{"1min":0.503,"5min":0.34499999999999997,"15min":0.32500000000000001},"max":{"1min":0.94099999999999995,"5min":1.224,"15min":1.224},"last":0.53300000000000003}]},{"osd":1,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.73399999999999999,"15min":0.64300000000000002},"min":{"1min":0.42699999999999999,"5min":0.251,"15min":0.251},"max":{"1min":1.452,"5min":1.452,"15min":1.452},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.70299999999999996},"min":{"1min":0.57499999999999996,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0289999999999999,"5min":1.361,"15min":1.4159999999999999},"last":0.48299999999999998}]},{"osd":3,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.80500000000000005,"15min":0.76700000000000002},"min":{"1min":0.53800000000000003,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.96899999999999997,"5min":0.94299999999999995,"15min":0.84399999999999997},"min":{"1min":0.63,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":0.72699999999999998}]},{"osd":4,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0489999999999999,"15min":0.98999999999999999},"min":{"1min":0.74299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":1.0029999999999999,"5min":0.97099999999999997,"15min":0.94099999999999995},"min":{"1min":0.86199999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.24,"5min":1.393,"15min":1.393},"last":0.83399999999999996}]},{"osd":5,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":1.1799999999999999,"5min":1.107,"15min":1.0569999999999999},"min":{"1min":0.86299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.722,"5min":1.722,"15min":1.722},"last":1.119},{"interface":"front","average":{"1min":1.321,"5min":1.2170000000000001,"15min":1.198},"min":{"1min":1.034,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6839999999999999,"5min":1.6839999999999999,"15min":1.734},"last":1.2669999999999999}]},{"osd":6,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.1399999999999999,"15min":1.1479999999999999},"min":{"1min":0.86199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.647,"5min":1.647,"15min":1.647},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.2210000000000001,"15min":1.2110000000000001},"min":{"1min":0.96599999999999997,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.631,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.167}]},{"osd":7,"last update":"Sun Feb 18 01:21:07 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.248,"15min":1.234},"min":{"1min":0.88,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.7669999999999999,"5min":1.7669999999999999,"15min":1.7669999999999999},"last":1.095},{"interface":"front","average":{"1min":1.181,"5min":1.1100000000000001,"15min":1.1060000000000001},"min":{"1min":0.875,"5min":0.53100000000000003,"15min":0.53100000000000003},"max":{"1min":1.4299999999999999,"5min":1.502,"15min":1.502},"last":1.0449999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182495,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297748,"kb_used_data":652,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447404,"statfs":{"total":95995035648,"available":95690141696,"internally_reserved":0,"allocated":667648,"data_stored":245351,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.68799999999999994,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.099,"5min":1.099,"15min":1.099},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.79900000000000004,"15min":0.77100000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.90200000000000002}]},{"osd":1,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.74099999999999999,"15min":0.68799999999999994},"min":{"1min":0.53200000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92300000000000004,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.72199999999999998,"15min":0.69199999999999995},"min":{"1min":0.39900000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.077,"5min":1.1319999999999999,"15min":1.1319999999999999},"last":0.85899999999999999}]},{"osd":2,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":0.92200000000000004,"5min":0.85999999999999999,"15min":0.82399999999999995},"min":{"1min":0.66200000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.1559999999999999,"5min":1.3,"15min":1.3},"last":1.054},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.76500000000000001},"min":{"1min":0.47499999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":1.155,"5min":1.27,"15min":1.286},"last":0.996}]},{"osd":4,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.92900000000000005,"15min":0.91100000000000003},"min":{"1min":0.64900000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.175,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.0409999999999999,"5min":1.002,"15min":0.94499999999999995},"min":{"1min":0.81299999999999994,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.407,"5min":1.6040000000000001,"15min":1.6040000000000001},"last":1.087}]},{"osd":5,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.0129999999999999,"15min":0.997},"min":{"1min":0.76900000000000002,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":1.163},{"interface":"front","average":{"1min":1.034,"5min":1.038,"15min":1.0349999999999999},"min":{"1min":0.75700000000000001,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3660000000000001,"5min":1.6020000000000001,"15min":1.6020000000000001},"last":1.2130000000000001}]},{"osd":6,"last update":"Sun Feb 18 01:21:22 2024","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.0900000000000001,"15min":1.097},"min":{"1min":0.95599999999999996,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.5820000000000001,"5min":1.625,"15min":1.625},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.1100000000000001,"15min":1.0820000000000001},"min":{"1min":0.82499999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3560000000000001,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.28}]},{"osd":7,"last update":"Sun Feb 18 01:21:00 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.1100000000000001,"15min":1.1020000000000001},"min":{"1min":0.63100000000000001,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.3540000000000001,"5min":1.625,"15min":1.625},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.2190000000000001,"5min":1.1379999999999999,"15min":1.1080000000000001},"min":{"1min":0.878,"5min":0.57499999999999996,"15min":0.57499999999999996},"max":{"1min":1.6739999999999999,"5min":1.748,"15min":1.748},"last":1.2569999999999999}]}]},{"osd":4,"up_from":31,"seq":133143986262,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297740,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447412,"statfs":{"total":95995035648,"available":95690149888,"internally_reserved":0,"allocated":659456,"data_stored":245377,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0449999999999999,"5min":0.90900000000000003,"15min":0.85999999999999999},"min":{"1min":0.58399999999999996,"5min":0.435,"15min":0.435},"max":{"1min":2.0499999999999998,"5min":2.0499999999999998,"15min":2.0499999999999998},"last":1.292},{"interface":"front","average":{"1min":1.002,"5min":0.91600000000000004,"15min":0.86699999999999999},"min":{"1min":0.68000000000000005,"5min":0.505,"15min":0.505},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.81200000000000006}]},{"osd":1,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.91400000000000003,"15min":0.85499999999999998},"min":{"1min":0.55400000000000005,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.1739999999999999,"5min":2.1739999999999999,"15min":2.1739999999999999},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.99399999999999999,"5min":0.94799999999999995,"15min":0.872},"min":{"1min":0.745,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.97,"5min":1.97,"15min":1.97},"last":0.89300000000000002}]},{"osd":2,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.92900000000000005,"15min":0.85599999999999998},"min":{"1min":0.749,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2649999999999999,"5min":1.806,"15min":1.806},"last":1.02},{"interface":"front","average":{"1min":1.0940000000000001,"5min":0.97299999999999998,"15min":0.95399999999999996},"min":{"1min":0.76500000000000001,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":2.024,"5min":2.024,"15min":2.024},"last":0.93600000000000005}]},{"osd":3,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":0.96799999999999997,"15min":0.94099999999999995},"min":{"1min":0.79900000000000004,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.6359999999999999,"5min":1.6359999999999999,"15min":1.6359999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.0640000000000001,"5min":0.98699999999999999,"15min":0.93999999999999995},"min":{"1min":0.76000000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":2.7629999999999999,"5min":2.7629999999999999,"15min":2.7629999999999999},"last":0.97599999999999998}]},{"osd":5,"last update":"Sun Feb 18 01:21:45 2024","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.92300000000000004,"15min":0.85199999999999998},"min":{"1min":0.64600000000000002,"5min":0.44,"15min":0.44},"max":{"1min":1.3320000000000001,"5min":2.177,"15min":2.177},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.159,"5min":1.0860000000000001,"15min":0.95299999999999996},"min":{"1min":0.83899999999999997,"5min":0.58699999999999997,"15min":0.58499999999999996},"max":{"1min":1.7030000000000001,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":1.2190000000000001}]},{"osd":6,"last update":"Sun Feb 18 01:21:30 2024","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":1.036,"15min":1.0760000000000001},"min":{"1min":0.77800000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.1659999999999999,"5min":3.2719999999999998,"15min":3.2719999999999998},"last":1.137},{"interface":"front","average":{"1min":1.075,"5min":1.0229999999999999,"15min":1.0429999999999999},"min":{"1min":0.67100000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6479999999999999,"5min":2.508,"15min":2.508},"last":1.0980000000000001}]},{"osd":7,"last update":"Sun Feb 18 01:21:03 2024","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.0609999999999999,"15min":1.0209999999999999},"min":{"1min":0.754,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.3859999999999999,"5min":2.6139999999999999,"15min":2.6139999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.073,"5min":1.0680000000000001,"15min":1.0680000000000001},"min":{"1min":0.79500000000000004,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.3260000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1060000000000001}]}]},{"osd":5,"up_from":37,"seq":158913790031,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297704,"kb_used_data":608,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":93447448,"statfs":{"total":95995035648,"available":95690186752,"internally_reserved":0,"allocated":622592,"data_stored":243309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.85699999999999998,"15min":0.82599999999999996},"min":{"1min":0.61699999999999999,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.21,"5min":1.248,"15min":1.248},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.86599999999999999,"15min":0.85599999999999998},"min":{"1min":0.66800000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.0880000000000001,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.081}]},{"osd":1,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.879,"15min":0.878},"min":{"1min":0.59299999999999997,"5min":0.5,"15min":0.5},"max":{"1min":1.331,"5min":1.331,"15min":1.331},"last":0.88},{"interface":"front","average":{"1min":0.878,"5min":0.89400000000000002,"15min":0.84599999999999997},"min":{"1min":0.64800000000000002,"5min":0.60599999999999998,"15min":0.59399999999999997},"max":{"1min":1.069,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.94599999999999995}]},{"osd":2,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.95599999999999996,"5min":0.90400000000000003,"15min":0.89700000000000002},"min":{"1min":0.66400000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.22,"5min":1.28,"15min":1.28},"last":0.99299999999999999},{"interface":"front","average":{"1min":1,"5min":0.94499999999999995,"15min":0.88100000000000001},"min":{"1min":0.628,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.0269999999999999}]},{"osd":3,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91800000000000004,"15min":0.91300000000000003},"min":{"1min":0.72099999999999997,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":1.109},{"interface":"front","average":{"1min":1.046,"5min":1.0149999999999999,"15min":0.96899999999999997},"min":{"1min":0.83799999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.302,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":1.121}]},{"osd":4,"last update":"Sun Feb 18 01:21:41 2024","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.88400000000000001,"15min":0.81200000000000006},"min":{"1min":0.625,"5min":0.47699999999999998,"15min":0.46899999999999997},"max":{"1min":1.1599999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":1.167},{"interface":"front","average":{"1min":1.087,"5min":1.032,"15min":0.93200000000000005},"min":{"1min":0.61099999999999999,"5min":0.53900000000000003,"15min":0.52700000000000002},"max":{"1min":1.3340000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.1359999999999999}]},{"osd":6,"last update":"Sun Feb 18 01:21:29 2024","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.94399999999999995,"15min":0.91600000000000004},"min":{"1min":0.60399999999999998,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.294,"5min":1.294,"15min":1.294},"last":1.069},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.002,"15min":0.95299999999999996},"min":{"1min":0.53000000000000003,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.411,"5min":1.423,"15min":1.423},"last":1.1819999999999999}]},{"osd":7,"last update":"Sun Feb 18 01:21:05 2024","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.01,"15min":1.028},"min":{"1min":0.76600000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.234,"5min":1.4199999999999999,"15min":1.4199999999999999},"last":1.234},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.0660000000000001,"15min":1.0640000000000001},"min":{"1min":0.69399999999999995,"5min":0.63,"15min":0.63},"max":{"1min":1.3640000000000001,"5min":1.371,"15min":1.371},"last":1.198}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":2228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":908,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-02-18T01:22:00.600 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-02-18T01:22:00.600 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-02-18T01:22:00.600 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-02-18T01:22:00.600 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81bd20d634209c7cb82c18be12b4b5a05643ebf1 shell --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 -- ceph health --format=json 2024-02-18T01:22:00.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:00 smithi155 bash[20805]: audit 2024-02-18T01:21:59.936246+0000 mgr.y (mgr.24475) 187 : audit [DBG] from='client.24680 v1:172.21.15.105:0/74174165' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:22:00.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:00 smithi105 bash[19734]: audit 2024-02-18T01:21:59.936246+0000 mgr.y (mgr.24475) 187 : audit [DBG] from='client.24680 v1:172.21.15.105:0/74174165' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:22:00.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:00 smithi105 bash[27669]: audit 2024-02-18T01:21:59.936246+0000 mgr.y (mgr.24475) 187 : audit [DBG] from='client.24680 v1:172.21.15.105:0/74174165' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-02-18T01:22:01.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:01 smithi155 bash[20805]: cluster 2024-02-18T01:22:00.387600+0000 mgr.y (mgr.24475) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:01.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:01 smithi105 bash[19734]: cluster 2024-02-18T01:22:00.387600+0000 mgr.y (mgr.24475) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:01 smithi105 bash[27669]: cluster 2024-02-18T01:22:00.387600+0000 mgr.y (mgr.24475) 188 : cluster [DBG] pgmap v137: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:03 smithi105 bash[19734]: cluster 2024-02-18T01:22:02.389006+0000 mgr.y (mgr.24475) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:03.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:03 smithi105 bash[27669]: cluster 2024-02-18T01:22:02.389006+0000 mgr.y (mgr.24475) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:03 smithi155 bash[20805]: cluster 2024-02-18T01:22:02.389006+0000 mgr.y (mgr.24475) 189 : cluster [DBG] pgmap v138: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:05.437 INFO:teuthology.orchestra.run.smithi105.stderr:Inferring config /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/mon.a/config 2024-02-18T01:22:05.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:05 smithi105 bash[19734]: cluster 2024-02-18T01:22:04.389767+0000 mgr.y (mgr.24475) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:05.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:05 smithi105 bash[27669]: cluster 2024-02-18T01:22:04.389767+0000 mgr.y (mgr.24475) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:05.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:05 smithi155 bash[20805]: cluster 2024-02-18T01:22:04.389767+0000 mgr.y (mgr.24475) 190 : cluster [DBG] pgmap v139: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:06] "GET /metrics HTTP/1.1" 200 172183 "" "Prometheus/2.33.4" 2024-02-18T01:22:07.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:07 smithi105 bash[19734]: cluster 2024-02-18T01:22:06.390433+0000 mgr.y (mgr.24475) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:07.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:07 smithi105 bash[27669]: cluster 2024-02-18T01:22:06.390433+0000 mgr.y (mgr.24475) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:07.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:07 smithi155 bash[20805]: cluster 2024-02-18T01:22:06.390433+0000 mgr.y (mgr.24475) 191 : cluster [DBG] pgmap v140: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:07.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:08.492 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- 172.21.15.105:0/1864335832 >> v1:172.21.15.105:6790/0 conn(0x7f51e00fc0b0 legacy=0x7f51e00fc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:22:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- 172.21.15.105:0/1864335832 shutdown_connections 2024-02-18T01:22:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- 172.21.15.105:0/1864335832 >> 172.21.15.105:0/1864335832 conn(0x7f51e006fef0 msgr2=0x7f51e00f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:22:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- 172.21.15.105:0/1864335832 shutdown_connections 2024-02-18T01:22:08.493 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- 172.21.15.105:0/1864335832 wait complete. 2024-02-18T01:22:08.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 Processor -- start 2024-02-18T01:22:08.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.486+0000 7f51e51ac700 1 -- start start 2024-02-18T01:22:08.495 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51e01194b0 con 0x7f51e00fef00 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51e011a6b0 con 0x7f51e0102a10 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f51e011b8b0 con 0x7f51e00fc0b0 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51de59c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f51e00fef00 0x7f51e0114460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:36506/0 (socket says 172.21.15.105:36506) 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51de59c700 1 -- 172.21.15.105:0/3187882365 learned_addr learned my addr 172.21.15.105:0/3187882365 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51ded9d700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f51e00fc0b0 0x7f51e00fb3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:49914/0 (socket says 172.21.15.105:49914) 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1611003739 0 0) 0x7f51e01194b0 con 0x7f51e00fef00 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51c8003030 con 0x7f51e00fef00 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1621330782 0 0) 0x7f51e011b8b0 con 0x7f51e00fc0b0 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e01194b0 con 0x7f51e00fc0b0 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369550312 0 0) 0x7f51e011a6b0 con 0x7f51e0102a10 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51e011b8b0 con 0x7f51e0102a10 2024-02-18T01:22:08.496 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3373533960 0 0) 0x7f51c8003030 con 0x7f51e00fef00 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51e011a6b0 con 0x7f51e00fef00 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f51d0001980 con 0x7f51e00fef00 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2044650368 0 0) 0x7f51e01194b0 con 0x7f51e00fc0b0 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51c8003030 con 0x7f51e00fc0b0 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f51d40015b0 con 0x7f51e00fc0b0 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2618027724 0 0) 0x7f51e011b8b0 con 0x7f51e0102a10 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f51e01194b0 con 0x7f51e0102a10 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f51d8001410 con 0x7f51e0102a10 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 552326053 0 0) 0x7f51e011a6b0 con 0x7f51e00fef00 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 >> v1:172.21.15.105:6790/0 conn(0x7f51e00fc0b0 legacy=0x7f51e00fb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 >> v1:172.21.15.155:6789/0 conn(0x7f51e0102a10 legacy=0x7f51e0117b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:22:08.497 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e011cab0 con 0x7f51e00fef00 2024-02-18T01:22:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e011baa0 con 0x7f51e00fef00 2024-02-18T01:22:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e011bf90 con 0x7f51e00fef00 2024-02-18T01:22:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f51d0001f80 con 0x7f51e00fef00 2024-02-18T01:22:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f51d00059d0 con 0x7f51e00fef00 2024-02-18T01:22:08.498 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51cc004f20 con 0x7f51e00fef00 2024-02-18T01:22:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.490+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f51d0001ae0 con 0x7f51e00fef00 2024-02-18T01:22:08.499 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.494+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(64..64 src has 1..64) v4 ==== 6143+0+0 (unknown 3105328921 0 0) 0x7f51d0078410 con 0x7f51e00fef00 2024-02-18T01:22:08.503 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.498+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f51d004bfc0 con 0x7f51e00fef00 2024-02-18T01:22:08.723 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.718+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f51cc005750 con 0x7f51e00fef00 2024-02-18T01:22:08.724 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.718+0000 7f51c7fff700 1 -- 172.21.15.105:0/3187882365 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (unknown 3730786198 0 3397129104) 0x7f51d0046e60 con 0x7f51e00fef00 2024-02-18T01:22:08.724 INFO:teuthology.orchestra.run.smithi105.stdout: 2024-02-18T01:22:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 >> v1:172.21.15.105:6800/2148176012 conn(0x7f51c8062290 legacy=0x7f51c8064750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:22:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 >> v1:172.21.15.105:6789/0 conn(0x7f51e00fef00 legacy=0x7f51e0114460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:22:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 shutdown_connections 2024-02-18T01:22:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 >> 172.21.15.105:0/3187882365 conn(0x7f51e006fef0 msgr2=0x7f51e00f4190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:22:08.726 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 shutdown_connections 2024-02-18T01:22:08.727 INFO:teuthology.orchestra.run.smithi105.stderr:2024-02-18T01:22:08.722+0000 7f51e51ac700 1 -- 172.21.15.105:0/3187882365 wait complete. 2024-02-18T01:22:08.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[27669]: audit 2024-02-18T01:22:07.516612+0000 mgr.y (mgr.24475) 192 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[27669]: audit 2024-02-18T01:22:08.157932+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[27669]: audit 2024-02-18T01:22:08.477447+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[27669]: audit 2024-02-18T01:22:08.478673+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[27669]: audit 2024-02-18T01:22:08.484110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[19734]: audit 2024-02-18T01:22:07.516612+0000 mgr.y (mgr.24475) 192 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[19734]: audit 2024-02-18T01:22:08.157932+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[19734]: audit 2024-02-18T01:22:08.477447+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[19734]: audit 2024-02-18T01:22:08.478673+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:22:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:08 smithi105 bash[19734]: audit 2024-02-18T01:22:08.484110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:22:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:08 smithi155 bash[20805]: audit 2024-02-18T01:22:07.516612+0000 mgr.y (mgr.24475) 192 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:08 smithi155 bash[20805]: audit 2024-02-18T01:22:08.157932+0000 mon.a (mon.0) 730 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:22:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:08 smithi155 bash[20805]: audit 2024-02-18T01:22:08.477447+0000 mon.a (mon.0) 731 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:22:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:08 smithi155 bash[20805]: audit 2024-02-18T01:22:08.478673+0000 mon.a (mon.0) 732 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:22:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:08 smithi155 bash[20805]: audit 2024-02-18T01:22:08.484110+0000 mon.a (mon.0) 733 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:22:09.384 INFO:teuthology.orchestra.run.smithi105.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-02-18T01:22:09.385 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-02-18T01:22:09.385 INFO:tasks.cephadm:Setup complete, yielding 2024-02-18T01:22:09.385 INFO:teuthology.run_tasks:Running task workunit... 2024-02-18T01:22:09.398 INFO:tasks.workunit:Pulling workunits from ref 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:22:09.399 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-02-18T01:22:09.399 DEBUG:teuthology.orchestra.run.smithi105:> stat -- /home/ubuntu/cephtest/mnt.0 2024-02-18T01:22:09.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T01:22:09.406 INFO:teuthology.orchestra.run.smithi105.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-02-18T01:22:09.406 DEBUG:teuthology.orchestra.run.smithi105:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-02-18T01:22:09.455 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-02-18T01:22:09.455 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-02-18T01:22:09.503 INFO:tasks.workunit:timeout=3h 2024-02-18T01:22:09.503 INFO:tasks.workunit:cleanup=True 2024-02-18T01:22:09.504 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 81bd20d634209c7cb82c18be12b4b5a05643ebf1 2024-02-18T01:22:09.552 INFO:tasks.workunit.client.0.smithi105.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-02-18T01:22:09.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:09 smithi105 bash[27669]: cluster 2024-02-18T01:22:08.391394+0000 mgr.y (mgr.24475) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:09.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:09 smithi105 bash[27669]: audit 2024-02-18T01:22:08.723449+0000 mon.a (mon.0) 734 : audit [DBG] from='client.? v1:172.21.15.105:0/3187882365' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T01:22:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:09 smithi105 bash[19734]: cluster 2024-02-18T01:22:08.391394+0000 mgr.y (mgr.24475) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:09.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:09 smithi105 bash[19734]: audit 2024-02-18T01:22:08.723449+0000 mon.a (mon.0) 734 : audit [DBG] from='client.? v1:172.21.15.105:0/3187882365' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T01:22:09.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:09 smithi155 bash[20805]: cluster 2024-02-18T01:22:08.391394+0000 mgr.y (mgr.24475) 193 : cluster [DBG] pgmap v141: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:09.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:09 smithi155 bash[20805]: audit 2024-02-18T01:22:08.723449+0000 mon.a (mon.0) 734 : audit [DBG] from='client.? v1:172.21.15.105:0/3187882365' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-02-18T01:22:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:11 smithi155 bash[20805]: cluster 2024-02-18T01:22:10.392059+0000 mgr.y (mgr.24475) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:11.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:11 smithi105 bash[19734]: cluster 2024-02-18T01:22:10.392059+0000 mgr.y (mgr.24475) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:11.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:11 smithi105 bash[27669]: cluster 2024-02-18T01:22:10.392059+0000 mgr.y (mgr.24475) 194 : cluster [DBG] pgmap v142: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:13 smithi105 bash[19734]: cluster 2024-02-18T01:22:12.393448+0000 mgr.y (mgr.24475) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:13 smithi105 bash[27669]: cluster 2024-02-18T01:22:12.393448+0000 mgr.y (mgr.24475) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:13.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:13 smithi155 bash[20805]: cluster 2024-02-18T01:22:12.393448+0000 mgr.y (mgr.24475) 195 : cluster [DBG] pgmap v143: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:15.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:15 smithi105 bash[27669]: cluster 2024-02-18T01:22:14.394117+0000 mgr.y (mgr.24475) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:15 smithi105 bash[19734]: cluster 2024-02-18T01:22:14.394117+0000 mgr.y (mgr.24475) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:15 smithi155 bash[20805]: cluster 2024-02-18T01:22:14.394117+0000 mgr.y (mgr.24475) 196 : cluster [DBG] pgmap v144: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:16] "GET /metrics HTTP/1.1" 200 172183 "" "Prometheus/2.33.4" 2024-02-18T01:22:17.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:17 smithi105 bash[19734]: cluster 2024-02-18T01:22:16.394789+0000 mgr.y (mgr.24475) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:17.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:17 smithi105 bash[27669]: cluster 2024-02-18T01:22:16.394789+0000 mgr.y (mgr.24475) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:17.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:17 smithi155 bash[20805]: cluster 2024-02-18T01:22:16.394789+0000 mgr.y (mgr.24475) 197 : cluster [DBG] pgmap v145: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:17.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:18.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:18 smithi105 bash[27669]: audit 2024-02-18T01:22:17.521560+0000 mgr.y (mgr.24475) 198 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:18 smithi105 bash[19734]: audit 2024-02-18T01:22:17.521560+0000 mgr.y (mgr.24475) 198 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:18 smithi155 bash[20805]: audit 2024-02-18T01:22:17.521560+0000 mgr.y (mgr.24475) 198 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:19.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:19 smithi105 bash[27669]: cluster 2024-02-18T01:22:18.395702+0000 mgr.y (mgr.24475) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:19.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:19 smithi105 bash[19734]: cluster 2024-02-18T01:22:18.395702+0000 mgr.y (mgr.24475) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:19 smithi155 bash[20805]: cluster 2024-02-18T01:22:18.395702+0000 mgr.y (mgr.24475) 199 : cluster [DBG] pgmap v146: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:21.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:21 smithi155 bash[20805]: cluster 2024-02-18T01:22:20.396301+0000 mgr.y (mgr.24475) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:21.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:21 smithi105 bash[27669]: cluster 2024-02-18T01:22:20.396301+0000 mgr.y (mgr.24475) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:21 smithi105 bash[19734]: cluster 2024-02-18T01:22:20.396301+0000 mgr.y (mgr.24475) 200 : cluster [DBG] pgmap v147: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:23.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:23 smithi105 bash[19734]: cluster 2024-02-18T01:22:22.397864+0000 mgr.y (mgr.24475) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:23 smithi105 bash[27669]: cluster 2024-02-18T01:22:22.397864+0000 mgr.y (mgr.24475) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:23 smithi155 bash[20805]: cluster 2024-02-18T01:22:22.397864+0000 mgr.y (mgr.24475) 201 : cluster [DBG] pgmap v148: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:25.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:25 smithi105 bash[27669]: cluster 2024-02-18T01:22:24.398371+0000 mgr.y (mgr.24475) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:25.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:25 smithi105 bash[19734]: cluster 2024-02-18T01:22:24.398371+0000 mgr.y (mgr.24475) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:25 smithi155 bash[20805]: cluster 2024-02-18T01:22:24.398371+0000 mgr.y (mgr.24475) 202 : cluster [DBG] pgmap v149: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:26] "GET /metrics HTTP/1.1" 200 172178 "" "Prometheus/2.33.4" 2024-02-18T01:22:27.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:27 smithi105 bash[19734]: cluster 2024-02-18T01:22:26.398937+0000 mgr.y (mgr.24475) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:27 smithi105 bash[27669]: cluster 2024-02-18T01:22:26.398937+0000 mgr.y (mgr.24475) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:27 smithi155 bash[20805]: cluster 2024-02-18T01:22:26.398937+0000 mgr.y (mgr.24475) 203 : cluster [DBG] pgmap v150: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:27.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:28 smithi105 bash[19734]: audit 2024-02-18T01:22:27.532686+0000 mgr.y (mgr.24475) 204 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:28 smithi105 bash[27669]: audit 2024-02-18T01:22:27.532686+0000 mgr.y (mgr.24475) 204 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:28 smithi155 bash[20805]: audit 2024-02-18T01:22:27.532686+0000 mgr.y (mgr.24475) 204 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:29.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:29 smithi105 bash[19734]: cluster 2024-02-18T01:22:28.399748+0000 mgr.y (mgr.24475) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:29 smithi105 bash[27669]: cluster 2024-02-18T01:22:28.399748+0000 mgr.y (mgr.24475) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:29.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:29 smithi155 bash[20805]: cluster 2024-02-18T01:22:28.399748+0000 mgr.y (mgr.24475) 205 : cluster [DBG] pgmap v151: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:31.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:31 smithi155 bash[20805]: cluster 2024-02-18T01:22:30.400166+0000 mgr.y (mgr.24475) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:31.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:31 smithi105 bash[19734]: cluster 2024-02-18T01:22:30.400166+0000 mgr.y (mgr.24475) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:31.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:31 smithi105 bash[27669]: cluster 2024-02-18T01:22:30.400166+0000 mgr.y (mgr.24475) 206 : cluster [DBG] pgmap v152: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:22:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:33.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:33 smithi105 bash[19734]: cluster 2024-02-18T01:22:32.400937+0000 mgr.y (mgr.24475) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:33 smithi105 bash[27669]: cluster 2024-02-18T01:22:32.400937+0000 mgr.y (mgr.24475) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:33 smithi155 bash[20805]: cluster 2024-02-18T01:22:32.400937+0000 mgr.y (mgr.24475) 207 : cluster [DBG] pgmap v153: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:35 smithi105 bash[19734]: cluster 2024-02-18T01:22:34.401349+0000 mgr.y (mgr.24475) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:35 smithi105 bash[27669]: cluster 2024-02-18T01:22:34.401349+0000 mgr.y (mgr.24475) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:35 smithi155 bash[20805]: cluster 2024-02-18T01:22:34.401349+0000 mgr.y (mgr.24475) 208 : cluster [DBG] pgmap v154: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:36] "GET /metrics HTTP/1.1" 200 172181 "" "Prometheus/2.33.4" 2024-02-18T01:22:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:37 smithi105 bash[19734]: cluster 2024-02-18T01:22:36.401749+0000 mgr.y (mgr.24475) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:37 smithi105 bash[27669]: cluster 2024-02-18T01:22:36.401749+0000 mgr.y (mgr.24475) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:37 smithi155 bash[20805]: cluster 2024-02-18T01:22:36.401749+0000 mgr.y (mgr.24475) 209 : cluster [DBG] pgmap v155: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:37.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:38 smithi105 bash[19734]: audit 2024-02-18T01:22:37.534174+0000 mgr.y (mgr.24475) 210 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:38 smithi105 bash[27669]: audit 2024-02-18T01:22:37.534174+0000 mgr.y (mgr.24475) 210 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:38.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:38 smithi155 bash[20805]: audit 2024-02-18T01:22:37.534174+0000 mgr.y (mgr.24475) 210 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:39 smithi105 bash[27669]: cluster 2024-02-18T01:22:38.402522+0000 mgr.y (mgr.24475) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:39 smithi105 bash[19734]: cluster 2024-02-18T01:22:38.402522+0000 mgr.y (mgr.24475) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:39.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:39 smithi155 bash[20805]: cluster 2024-02-18T01:22:38.402522+0000 mgr.y (mgr.24475) 211 : cluster [DBG] pgmap v156: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:41.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:41 smithi155 bash[20805]: cluster 2024-02-18T01:22:40.402928+0000 mgr.y (mgr.24475) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:41.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:41 smithi105 bash[27669]: cluster 2024-02-18T01:22:40.402928+0000 mgr.y (mgr.24475) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:41.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:41 smithi105 bash[19734]: cluster 2024-02-18T01:22:40.402928+0000 mgr.y (mgr.24475) 212 : cluster [DBG] pgmap v157: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:43 smithi105 bash[19734]: cluster 2024-02-18T01:22:42.403700+0000 mgr.y (mgr.24475) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:43 smithi105 bash[27669]: cluster 2024-02-18T01:22:42.403700+0000 mgr.y (mgr.24475) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:43.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:43 smithi155 bash[20805]: cluster 2024-02-18T01:22:42.403700+0000 mgr.y (mgr.24475) 213 : cluster [DBG] pgmap v158: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:45 smithi105 bash[19734]: cluster 2024-02-18T01:22:44.404110+0000 mgr.y (mgr.24475) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:45 smithi105 bash[27669]: cluster 2024-02-18T01:22:44.404110+0000 mgr.y (mgr.24475) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:45.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:45 smithi155 bash[20805]: cluster 2024-02-18T01:22:44.404110+0000 mgr.y (mgr.24475) 214 : cluster [DBG] pgmap v159: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:46] "GET /metrics HTTP/1.1" 200 172181 "" "Prometheus/2.33.4" 2024-02-18T01:22:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:47 smithi105 bash[27669]: cluster 2024-02-18T01:22:46.404519+0000 mgr.y (mgr.24475) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:47 smithi105 bash[19734]: cluster 2024-02-18T01:22:46.404519+0000 mgr.y (mgr.24475) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:47 smithi155 bash[20805]: cluster 2024-02-18T01:22:46.404519+0000 mgr.y (mgr.24475) 215 : cluster [DBG] pgmap v160: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:47.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:48 smithi105 bash[19734]: audit 2024-02-18T01:22:47.545409+0000 mgr.y (mgr.24475) 216 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:48 smithi105 bash[27669]: audit 2024-02-18T01:22:47.545409+0000 mgr.y (mgr.24475) 216 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:48 smithi155 bash[20805]: audit 2024-02-18T01:22:47.545409+0000 mgr.y (mgr.24475) 216 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:49 smithi105 bash[27669]: cluster 2024-02-18T01:22:48.405312+0000 mgr.y (mgr.24475) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:49 smithi105 bash[19734]: cluster 2024-02-18T01:22:48.405312+0000 mgr.y (mgr.24475) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:49.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:49 smithi155 bash[20805]: cluster 2024-02-18T01:22:48.405312+0000 mgr.y (mgr.24475) 217 : cluster [DBG] pgmap v161: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:51.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:51 smithi105 bash[27669]: cluster 2024-02-18T01:22:50.405929+0000 mgr.y (mgr.24475) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:51.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:51 smithi105 bash[19734]: cluster 2024-02-18T01:22:50.405929+0000 mgr.y (mgr.24475) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:51.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:51 smithi155 bash[20805]: cluster 2024-02-18T01:22:50.405929+0000 mgr.y (mgr.24475) 218 : cluster [DBG] pgmap v162: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:51.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:22:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:22:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:53 smithi105 bash[27669]: cluster 2024-02-18T01:22:52.407318+0000 mgr.y (mgr.24475) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:53 smithi105 bash[19734]: cluster 2024-02-18T01:22:52.407318+0000 mgr.y (mgr.24475) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:53.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:53 smithi155 bash[20805]: cluster 2024-02-18T01:22:52.407318+0000 mgr.y (mgr.24475) 219 : cluster [DBG] pgmap v163: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:55.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:55 smithi105 bash[19734]: cluster 2024-02-18T01:22:54.407972+0000 mgr.y (mgr.24475) 220 : cluster [DBG] pgmap v164: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:55 smithi105 bash[27669]: cluster 2024-02-18T01:22:54.407972+0000 mgr.y (mgr.24475) 220 : cluster [DBG] pgmap v164: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:55.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:55 smithi155 bash[20805]: cluster 2024-02-18T01:22:54.407972+0000 mgr.y (mgr.24475) 220 : cluster [DBG] pgmap v164: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:56.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:22:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:22:56] "GET /metrics HTTP/1.1" 200 172190 "" "Prometheus/2.33.4" 2024-02-18T01:22:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:57 smithi105 bash[19734]: cluster 2024-02-18T01:22:56.408715+0000 mgr.y (mgr.24475) 221 : cluster [DBG] pgmap v165: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:57 smithi105 bash[27669]: cluster 2024-02-18T01:22:56.408715+0000 mgr.y (mgr.24475) 221 : cluster [DBG] pgmap v165: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:57.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:22:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:22:57.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:57 smithi155 bash[20805]: cluster 2024-02-18T01:22:56.408715+0000 mgr.y (mgr.24475) 221 : cluster [DBG] pgmap v165: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:Note: checking out '81bd20d634209c7cb82c18be12b4b5a05643ebf1'. 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:changes and commit them, and you can discard any commits you make in this 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:state without impacting any branches by performing another checkout. 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:If you want to create a new branch to retain commits you create, you may 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr:do so (now or later) by using -b with the checkout command again. Example: 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-02-18T01:22:58.823 INFO:tasks.workunit.client.0.smithi105.stderr: git checkout -b 2024-02-18T01:22:58.824 INFO:tasks.workunit.client.0.smithi105.stderr: 2024-02-18T01:22:58.824 INFO:tasks.workunit.client.0.smithi105.stderr:HEAD is now at 81bd20d6342 Merge pull request #55523 from ljflores/wip-tracker-64343-pacific 2024-02-18T01:22:58.828 DEBUG:teuthology.orchestra.run.smithi105:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-02-18T01:22:58.879 INFO:tasks.workunit.client.0.smithi105.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-02-18T01:22:58.883 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T01:22:58.883 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-02-18T01:22:58.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:58 smithi105 bash[19734]: audit 2024-02-18T01:22:57.548419+0000 mgr.y (mgr.24475) 222 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:58 smithi105 bash[27669]: audit 2024-02-18T01:22:57.548419+0000 mgr.y (mgr.24475) 222 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:58 smithi155 bash[20805]: audit 2024-02-18T01:22:57.548419+0000 mgr.y (mgr.24475) 222 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:22:59.594 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-02-18T01:22:59.707 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-02-18T01:22:59.758 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-02-18T01:22:59.759 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T01:22:59.759 INFO:tasks.workunit.client.0.smithi105.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-02-18T01:22:59.809 INFO:tasks.workunit.client.0.smithi105.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-02-18T01:22:59.813 DEBUG:teuthology.orchestra.run.smithi105:> set -ex 2024-02-18T01:22:59.813 DEBUG:teuthology.orchestra.run.smithi105:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-02-18T01:22:59.864 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-02-18T01:22:59.865 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-02-18T01:22:59.865 DEBUG:teuthology.orchestra.run.smithi105:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81bd20d634209c7cb82c18be12b4b5a05643ebf1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-02-18T01:22:59.919 INFO:tasks.workunit.client.0.smithi105.stderr:+ parallel=1 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' '' = --serial ']' 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ color= 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' -t 1 ']' 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ declare -A pids 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.920 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.921 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_aio 2024-02-18T01:22:59.921 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_aio' 2024-02-18T01:22:59.921 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_aio 2024-02-18T01:22:59.921 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.923 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_aio 2024-02-18T01:22:59.923 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61644 2024-02-18T01:22:59.924 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_aio on pid 61644' 2024-02-18T01:22:59.924 INFO:tasks.workunit.client.0.smithi105.stdout:test api_aio on pid 61644 2024-02-18T01:22:59.924 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61644 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_aio_pp 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_aio_pp' 2024-02-18T01:22:59.925 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_aio_pp 2024-02-18T01:22:59.926 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.926 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_aio 2024-02-18T01:22:59.926 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_aio.log 2024-02-18T01:22:59.926 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_aio: /' 2024-02-18T01:22:59.928 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_aio_pp 2024-02-18T01:22:59.929 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61652 2024-02-18T01:22:59.929 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_aio_pp on pid 61652' 2024-02-18T01:22:59.929 INFO:tasks.workunit.client.0.smithi105.stdout:test api_aio_pp on pid 61652 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61652 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_io 2024-02-18T01:22:59.930 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_io' 2024-02-18T01:22:59.931 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_io 2024-02-18T01:22:59.931 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.931 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_aio_pp 2024-02-18T01:22:59.931 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-02-18T01:22:59.932 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_aio_pp: /' 2024-02-18T01:22:59.934 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_io 2024-02-18T01:22:59.935 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61660 2024-02-18T01:22:59.935 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_io on pid 61660' 2024-02-18T01:22:59.935 INFO:tasks.workunit.client.0.smithi105.stdout:test api_io on pid 61660 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61660 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_io_pp 2024-02-18T01:22:59.936 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_io_pp' 2024-02-18T01:22:59.937 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_io_pp 2024-02-18T01:22:59.937 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.937 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_io 2024-02-18T01:22:59.937 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_io.log 2024-02-18T01:22:59.937 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_io: /' 2024-02-18T01:22:59.939 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_io_pp 2024-02-18T01:22:59.939 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61668 2024-02-18T01:22:59.939 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_io_pp on pid 61668' 2024-02-18T01:22:59.939 INFO:tasks.workunit.client.0.smithi105.stdout:test api_io_pp on pid 61668 2024-02-18T01:22:59.943 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61668 2024-02-18T01:22:59.943 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.944 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.944 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-02-18T01:22:59.944 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_asio 2024-02-18T01:22:59.944 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_asio' 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_io_pp 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_asio 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_io_pp: /' 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_asio 2024-02-18T01:22:59.945 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61676 2024-02-18T01:22:59.946 INFO:tasks.workunit.client.0.smithi105.stdout:test api_asio on pid 61676 2024-02-18T01:22:59.946 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_asio on pid 61676' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61676 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_list 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_list' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_list 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_asio 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_asio: /' 2024-02-18T01:22:59.947 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_asio.log 2024-02-18T01:22:59.948 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_list 2024-02-18T01:22:59.948 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61684 2024-02-18T01:22:59.948 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_list on pid 61684' 2024-02-18T01:22:59.948 INFO:tasks.workunit.client.0.smithi105.stdout:test api_list on pid 61684 2024-02-18T01:22:59.949 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61684 2024-02-18T01:22:59.949 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.949 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.949 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-02-18T01:22:59.949 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_lock 2024-02-18T01:22:59.950 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_lock' 2024-02-18T01:22:59.950 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_lock 2024-02-18T01:22:59.950 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.951 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_list.log 2024-02-18T01:22:59.952 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_list 2024-02-18T01:22:59.952 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_list: /' 2024-02-18T01:22:59.953 INFO:tasks.workunit.client.0.smithi105.stdout:test api_lock on pid 61692 2024-02-18T01:22:59.953 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_lock 2024-02-18T01:22:59.953 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61692 2024-02-18T01:22:59.953 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_lock on pid 61692' 2024-02-18T01:22:59.954 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61692 2024-02-18T01:22:59.954 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.954 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.954 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-02-18T01:22:59.954 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_lock_pp 2024-02-18T01:22:59.955 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_lock_pp' 2024-02-18T01:22:59.955 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_lock_pp 2024-02-18T01:22:59.955 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.955 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_lock.log 2024-02-18T01:22:59.955 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_lock: /' 2024-02-18T01:22:59.956 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_lock 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stdout:test api_lock_pp on pid 61700 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_lock_pp 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61700 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_lock_pp on pid 61700' 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61700 2024-02-18T01:22:59.957 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.958 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.958 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_misc 2024-02-18T01:22:59.958 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_misc' 2024-02-18T01:22:59.959 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_misc 2024-02-18T01:22:59.959 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stdout:test api_misc on pid 61705 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_misc 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61705 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_misc on pid 61705' 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61705 2024-02-18T01:22:59.962 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.963 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.963 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-02-18T01:22:59.963 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-02-18T01:22:59.964 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_misc 2024-02-18T01:22:59.964 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_misc.log 2024-02-18T01:22:59.964 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_lock_pp: /' 2024-02-18T01:22:59.965 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_lock_pp 2024-02-18T01:22:59.969 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_misc_pp 2024-02-18T01:22:59.969 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_misc_pp' 2024-02-18T01:22:59.970 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_misc: /' 2024-02-18T01:22:59.973 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_misc_pp 2024-02-18T01:22:59.973 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-02-18T01:22:59.974 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.980 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_misc_pp 2024-02-18T01:22:59.980 INFO:tasks.workunit.client.0.smithi105.stdout:test api_misc_pp on pid 61765 2024-02-18T01:22:59.981 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61765 2024-02-18T01:22:59.981 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_misc_pp on pid 61765' 2024-02-18T01:22:59.981 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61765 2024-02-18T01:22:59.981 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.981 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.983 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-02-18T01:22:59.983 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_tier_pp 2024-02-18T01:22:59.983 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_tier_pp' 2024-02-18T01:22:59.987 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_tier_pp 2024-02-18T01:22:59.987 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:22:59.987 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_misc_pp 2024-02-18T01:22:59.988 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-02-18T01:22:59.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:22:59 smithi105 bash[19734]: cluster 2024-02-18T01:22:58.410247+0000 mgr.y (mgr.24475) 223 : cluster [DBG] pgmap v166: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:59.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:22:59 smithi105 bash[27669]: cluster 2024-02-18T01:22:58.410247+0000 mgr.y (mgr.24475) 223 : cluster [DBG] pgmap v166: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:59.990 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_misc_pp: /' 2024-02-18T01:22:59.994 INFO:tasks.workunit.client.0.smithi105.stdout:test api_tier_pp on pid 61805 2024-02-18T01:22:59.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:22:59 smithi155 bash[20805]: cluster 2024-02-18T01:22:58.410247+0000 mgr.y (mgr.24475) 223 : cluster [DBG] pgmap v166: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:22:59.994 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_tier_pp 2024-02-18T01:22:59.994 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61805 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_tier_pp on pid 61805' 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61805 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_pool 2024-02-18T01:22:59.995 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_pool' 2024-02-18T01:22:59.998 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-02-18T01:23:00.000 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_pool 2024-02-18T01:23:00.001 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.007 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_tier_pp 2024-02-18T01:23:00.007 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_pool 2024-02-18T01:23:00.007 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-02-18T01:23:00.007 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61845 2024-02-18T01:23:00.008 INFO:tasks.workunit.client.0.smithi105.stdout:test api_pool on pid 61845 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_pool on pid 61845' 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61845 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-02-18T01:23:00.009 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_tier_pp: /' 2024-02-18T01:23:00.012 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_pool: /' 2024-02-18T01:23:00.012 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_pool.log 2024-02-18T01:23:00.012 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_snapshots 2024-02-18T01:23:00.012 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_snapshots' 2024-02-18T01:23:00.012 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_pool 2024-02-18T01:23:00.029 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_snapshots 2024-02-18T01:23:00.029 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.032 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_snapshots 2024-02-18T01:23:00.032 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61889 2024-02-18T01:23:00.032 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_snapshots on pid 61889' 2024-02-18T01:23:00.032 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-02-18T01:23:00.033 INFO:tasks.workunit.client.0.smithi105.stdout:test api_snapshots on pid 61889 2024-02-18T01:23:00.033 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61889 2024-02-18T01:23:00.033 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.033 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.033 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_snapshots: /' 2024-02-18T01:23:00.034 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_snapshots 2024-02-18T01:23:00.034 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_snapshots_pp 2024-02-18T01:23:00.034 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_snapshots_pp' 2024-02-18T01:23:00.037 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-02-18T01:23:00.041 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_snapshots_pp 2024-02-18T01:23:00.042 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.045 INFO:tasks.workunit.client.0.smithi105.stdout:test api_snapshots_pp on pid 61910 2024-02-18T01:23:00.045 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_snapshots_pp 2024-02-18T01:23:00.045 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61910 2024-02-18T01:23:00.045 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_snapshots_pp on pid 61910' 2024-02-18T01:23:00.045 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61910 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_stat 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_stat' 2024-02-18T01:23:00.046 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_snapshots_pp 2024-02-18T01:23:00.047 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.048 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_stat 2024-02-18T01:23:00.048 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-02-18T01:23:00.050 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-02-18T01:23:00.055 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_stat 2024-02-18T01:23:00.055 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61933 2024-02-18T01:23:00.055 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_stat on pid 61933' 2024-02-18T01:23:00.056 INFO:tasks.workunit.client.0.smithi105.stdout:test api_stat on pid 61933 2024-02-18T01:23:00.056 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61933 2024-02-18T01:23:00.056 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.056 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.057 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-02-18T01:23:00.057 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_stat_pp 2024-02-18T01:23:00.058 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_stat_pp' 2024-02-18T01:23:00.058 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.059 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_stat_pp 2024-02-18T01:23:00.059 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_stat_pp 2024-02-18T01:23:00.060 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61944 2024-02-18T01:23:00.060 INFO:tasks.workunit.client.0.smithi105.stdout:test api_stat_pp on pid 61944 2024-02-18T01:23:00.060 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_stat_pp on pid 61944' 2024-02-18T01:23:00.060 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61944 2024-02-18T01:23:00.060 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.061 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.061 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_stat_pp: /' 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_watch_notify 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_watch_notify' 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.062 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_watch_notify 2024-02-18T01:23:00.064 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_watch_notify 2024-02-18T01:23:00.064 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61955 2024-02-18T01:23:00.064 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_watch_notify on pid 61955' 2024-02-18T01:23:00.064 INFO:tasks.workunit.client.0.smithi105.stdout:test api_watch_notify on pid 61955 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61955 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_watch_notify: /' 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_watch_notify_pp 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_watch_notify 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_watch_notify_pp' 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_stat 2024-02-18T01:23:00.066 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_stat.log 2024-02-18T01:23:00.067 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_stat: /' 2024-02-18T01:23:00.067 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.067 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_watch_notify_pp 2024-02-18T01:23:00.067 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_stat_pp 2024-02-18T01:23:00.070 INFO:tasks.workunit.client.0.smithi105.stdout:test api_watch_notify_pp on pid 61970 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_watch_notify_pp 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61970 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_watch_notify_pp on pid 61970' 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61970 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-02-18T01:23:00.071 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_cmd 2024-02-18T01:23:00.072 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_cmd' 2024-02-18T01:23:00.074 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-02-18T01:23:00.074 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-02-18T01:23:00.074 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-02-18T01:23:00.074 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_cmd 2024-02-18T01:23:00.075 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.077 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_cmd 2024-02-18T01:23:00.078 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61985 2024-02-18T01:23:00.078 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_cmd on pid 61985' 2024-02-18T01:23:00.078 INFO:tasks.workunit.client.0.smithi105.stdout:test api_cmd on pid 61985 2024-02-18T01:23:00.078 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=61985 2024-02-18T01:23:00.079 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.079 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.085 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-02-18T01:23:00.086 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_cmd_pp 2024-02-18T01:23:00.086 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_cmd_pp' 2024-02-18T01:23:00.097 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_cmd 2024-02-18T01:23:00.098 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_cmd.log 2024-02-18T01:23:00.100 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_cmd_pp 2024-02-18T01:23:00.102 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_cmd: /' 2024-02-18T01:23:00.104 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.107 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_cmd_pp 2024-02-18T01:23:00.107 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62045 2024-02-18T01:23:00.107 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_cmd_pp on pid 62045' 2024-02-18T01:23:00.107 INFO:tasks.workunit.client.0.smithi105.stdout:test api_cmd_pp on pid 62045 2024-02-18T01:23:00.107 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62045 2024-02-18T01:23:00.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-02-18T01:23:00.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-02-18T01:23:00.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_cmd_pp 2024-02-18T01:23:00.109 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_service 2024-02-18T01:23:00.109 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_service' 2024-02-18T01:23:00.113 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-02-18T01:23:00.116 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_service 2024-02-18T01:23:00.121 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.124 INFO:tasks.workunit.client.0.smithi105.stdout:test api_service on pid 62088 2024-02-18T01:23:00.125 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_service 2024-02-18T01:23:00.126 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62088 2024-02-18T01:23:00.126 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_service on pid 62088' 2024-02-18T01:23:00.126 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62088 2024-02-18T01:23:00.126 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.126 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.129 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-02-18T01:23:00.133 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_service.log 2024-02-18T01:23:00.133 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_service: /' 2024-02-18T01:23:00.133 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_service_pp 2024-02-18T01:23:00.133 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_service_pp' 2024-02-18T01:23:00.133 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_service 2024-02-18T01:23:00.134 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.135 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_service_pp 2024-02-18T01:23:00.136 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_service_pp 2024-02-18T01:23:00.136 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62125 2024-02-18T01:23:00.136 INFO:tasks.workunit.client.0.smithi105.stdout:test api_service_pp on pid 62125 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_service_pp on pid 62125' 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62125 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_c_write_operations 2024-02-18T01:23:00.137 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_c_write_operations' 2024-02-18T01:23:00.138 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.138 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_c_write_operations 2024-02-18T01:23:00.139 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_c_write_operations 2024-02-18T01:23:00.139 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62133 2024-02-18T01:23:00.139 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_c_write_operations on pid 62133' 2024-02-18T01:23:00.139 INFO:tasks.workunit.client.0.smithi105.stdout:test api_c_write_operations on pid 62133 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62133 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s api_c_read_operations 2024-02-18T01:23:00.140 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' api_c_read_operations' 2024-02-18T01:23:00.141 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_c_write_operations 2024-02-18T01:23:00.141 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-02-18T01:23:00.141 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-02-18T01:23:00.141 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo api_c_read_operations 2024-02-18T01:23:00.142 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.144 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=api_c_read_operations 2024-02-18T01:23:00.144 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-02-18T01:23:00.147 INFO:tasks.workunit.client.0.smithi105.stdout:test api_c_read_operations on pid 62145 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62145 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test api_c_read_operations on pid 62145' 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62145 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_c_read_operations 2024-02-18T01:23:00.158 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-02-18T01:23:00.158 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s list_parallel 2024-02-18T01:23:00.158 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' list_parallel' 2024-02-18T01:23:00.159 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-02-18T01:23:00.162 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_api_service_pp 2024-02-18T01:23:00.162 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_service_pp: /' 2024-02-18T01:23:00.163 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo list_parallel 2024-02-18T01:23:00.163 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.164 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-02-18T01:23:00.166 INFO:tasks.workunit.client.0.smithi105.stdout:test list_parallel on pid 62166 2024-02-18T01:23:00.166 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=list_parallel 2024-02-18T01:23:00.166 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62166 2024-02-18T01:23:00.166 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test list_parallel on pid 62166' 2024-02-18T01:23:00.166 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62166 2024-02-18T01:23:00.167 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.167 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.169 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-02-18T01:23:00.170 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-02-18T01:23:00.170 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s open_pools_parallel 2024-02-18T01:23:00.171 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' open_pools_parallel' 2024-02-18T01:23:00.175 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_list_parallel 2024-02-18T01:23:00.177 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_list_parallel.log 2024-02-18T01:23:00.178 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ list_parallel: /' 2024-02-18T01:23:00.185 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo open_pools_parallel 2024-02-18T01:23:00.197 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.200 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=open_pools_parallel 2024-02-18T01:23:00.200 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62245 2024-02-18T01:23:00.200 INFO:tasks.workunit.client.0.smithi105.stdout:test open_pools_parallel on pid 62245 2024-02-18T01:23:00.201 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test open_pools_parallel on pid 62245' 2024-02-18T01:23:00.201 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62245 2024-02-18T01:23:00.201 INFO:tasks.workunit.client.0.smithi105.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations list_parallel open_pools_parallel delete_pools_parallel 2024-02-18T01:23:00.201 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.208 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-02-18T01:23:00.209 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-02-18T01:23:00.210 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_open_pools_parallel 2024-02-18T01:23:00.213 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-02-18T01:23:00.213 INFO:tasks.workunit.client.0.smithi105.stderr:++ printf %25s delete_pools_parallel 2024-02-18T01:23:00.214 INFO:tasks.workunit.client.0.smithi105.stderr:+ r=' delete_pools_parallel' 2024-02-18T01:23:00.227 INFO:tasks.workunit.client.0.smithi105.stderr:++ awk '{print $1}' 2024-02-18T01:23:00.233 INFO:tasks.workunit.client.0.smithi105.stderr:++ echo delete_pools_parallel 2024-02-18T01:23:00.234 INFO:tasks.workunit.client.0.smithi105.stderr:+ ff=delete_pools_parallel 2024-02-18T01:23:00.234 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62321 2024-02-18T01:23:00.234 INFO:tasks.workunit.client.0.smithi105.stdout:test delete_pools_parallel on pid 62321 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo 'test delete_pools_parallel on pid 62321' 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ pids[$f]=62321 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ ret=0 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ '[' 1 -eq 1 ']' 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:23:00.235 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61889 2024-02-18T01:23:00.236 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61889 2024-02-18T01:23:00.236 INFO:tasks.workunit.client.0.smithi105.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-02-18T01:23:00.236 INFO:tasks.workunit.client.0.smithi105.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-02-18T01:23:00.236 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph_test_rados_delete_pools_parallel 2024-02-18T01:23:00.241 INFO:tasks.workunit.client.0.smithi105.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-02-18T01:23:02.179 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:02.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:02 smithi105 bash[27669]: audit 2024-02-18T01:23:00.128140+0000 mon.c (mon.2) 31 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:02.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:02 smithi105 bash[27669]: audit 2024-02-18T01:23:00.144144+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? v1:172.21.15.105:0/584819234' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T01:23:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:02 smithi105 bash[19734]: audit 2024-02-18T01:23:00.128140+0000 mon.c (mon.2) 31 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:02 smithi105 bash[19734]: audit 2024-02-18T01:23:00.144144+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? v1:172.21.15.105:0/584819234' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T01:23:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:02 smithi155 bash[20805]: audit 2024-02-18T01:23:00.128140+0000 mon.c (mon.2) 31 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:02.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:02 smithi155 bash[20805]: audit 2024-02-18T01:23:00.144144+0000 mon.c (mon.2) 32 : audit [DBG] from='client.? v1:172.21.15.105:0/584819234' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-02-18T01:23:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:06] "GET /metrics HTTP/1.1" 200 184464 "" "Prometheus/2.33.4" 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: cluster 2024-02-18T01:23:00.410976+0000 mgr.y (mgr.24475) 224 : cluster [DBG] pgmap v167: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: audit 2024-02-18T01:23:00.891795+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.105:0/192578380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: cluster 2024-02-18T01:23:01.744619+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: audit 2024-02-18T01:23:02.182016+0000 mon.a (mon.0) 736 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: audit 2024-02-18T01:23:02.183068+0000 mon.a (mon.0) 737 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:07.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[19734]: audit 2024-02-18T01:23:02.188149+0000 mon.a (mon.0) 738 : audit [INF] from='client.24707 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: cluster 2024-02-18T01:23:00.410976+0000 mgr.y (mgr.24475) 224 : cluster [DBG] pgmap v167: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: audit 2024-02-18T01:23:00.891795+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.105:0/192578380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: cluster 2024-02-18T01:23:01.744619+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: audit 2024-02-18T01:23:02.182016+0000 mon.a (mon.0) 736 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: audit 2024-02-18T01:23:02.183068+0000 mon.a (mon.0) 737 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:07.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:07 smithi105 bash[27669]: audit 2024-02-18T01:23:02.188149+0000 mon.a (mon.0) 738 : audit [INF] from='client.24707 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.744 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: cluster 2024-02-18T01:23:00.410976+0000 mgr.y (mgr.24475) 224 : cluster [DBG] pgmap v167: 132 pgs: 132 active+clean; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: audit 2024-02-18T01:23:00.891795+0000 mon.c (mon.2) 33 : audit [INF] from='client.? v1:172.21.15.105:0/192578380' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: cluster 2024-02-18T01:23:01.744619+0000 mon.a (mon.0) 735 : cluster [DBG] osdmap e65: 8 total, 8 up, 8 in 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: audit 2024-02-18T01:23:02.182016+0000 mon.a (mon.0) 736 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: audit 2024-02-18T01:23:02.183068+0000 mon.a (mon.0) 737 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:07.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:07 smithi155 bash[20805]: audit 2024-02-18T01:23:02.188149+0000 mon.a (mon.0) 738 : audit [INF] from='client.24707 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: audit 2024-02-18T01:23:02.197451+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.105:0/4040232489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: audit 2024-02-18T01:23:02.199678+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.105:0/354534175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: cluster 2024-02-18T01:23:02.413167+0000 mgr.y (mgr.24475) 225 : cluster [DBG] pgmap v170: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: cluster 2024-02-18T01:23:04.413967+0000 mgr.y (mgr.24475) 226 : cluster [DBG] pgmap v171: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: audit 2024-02-18T01:23:05.479737+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? v1:172.21.15.105:0/609554011' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: cluster 2024-02-18T01:23:06.414800+0000 mgr.y (mgr.24475) 227 : cluster [DBG] pgmap v172: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: cluster 2024-02-18T01:23:07.336723+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: audit 2024-02-18T01:23:07.349257+0000 mon.a (mon.0) 740 : audit [INF] from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[27669]: audit 2024-02-18T01:23:07.349910+0000 mon.a (mon.0) 741 : audit [INF] from='client.24713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: audit 2024-02-18T01:23:02.197451+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.105:0/4040232489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: audit 2024-02-18T01:23:02.199678+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.105:0/354534175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: cluster 2024-02-18T01:23:02.413167+0000 mgr.y (mgr.24475) 225 : cluster [DBG] pgmap v170: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: cluster 2024-02-18T01:23:04.413967+0000 mgr.y (mgr.24475) 226 : cluster [DBG] pgmap v171: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: audit 2024-02-18T01:23:05.479737+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? v1:172.21.15.105:0/609554011' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: cluster 2024-02-18T01:23:06.414800+0000 mgr.y (mgr.24475) 227 : cluster [DBG] pgmap v172: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: cluster 2024-02-18T01:23:07.336723+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: audit 2024-02-18T01:23:07.349257+0000 mon.a (mon.0) 740 : audit [INF] from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:10 smithi105 bash[19734]: audit 2024-02-18T01:23:07.349910+0000 mon.a (mon.0) 741 : audit [INF] from='client.24713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: audit 2024-02-18T01:23:02.197451+0000 mon.c (mon.2) 34 : audit [INF] from='client.? v1:172.21.15.105:0/4040232489' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: audit 2024-02-18T01:23:02.199678+0000 mon.b (mon.1) 27 : audit [INF] from='client.? v1:172.21.15.105:0/354534175' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: cluster 2024-02-18T01:23:02.413167+0000 mgr.y (mgr.24475) 225 : cluster [DBG] pgmap v170: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: cluster 2024-02-18T01:23:04.413967+0000 mgr.y (mgr.24475) 226 : cluster [DBG] pgmap v171: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: audit 2024-02-18T01:23:05.479737+0000 mon.c (mon.2) 35 : audit [DBG] from='client.? v1:172.21.15.105:0/609554011' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: cluster 2024-02-18T01:23:06.414800+0000 mgr.y (mgr.24475) 227 : cluster [DBG] pgmap v172: 292 pgs: 150 creating+peering, 132 active+clean, 10 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: cluster 2024-02-18T01:23:07.336723+0000 mon.a (mon.0) 739 : cluster [DBG] osdmap e66: 8 total, 8 up, 8 in 2024-02-18T01:23:10.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: audit 2024-02-18T01:23:07.349257+0000 mon.a (mon.0) 740 : audit [INF] from='client.24832 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:10.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:10 smithi155 bash[20805]: audit 2024-02-18T01:23:07.349910+0000 mon.a (mon.0) 741 : audit [INF] from='client.24713 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: Running main() from gmock_main.cc 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [----------] Global test environment set-up. 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (38 ms) 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (41 ms) 2024-02-18T01:23:11.507 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (11245 ms) 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.357532+0000 mon.a [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.363886+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/799611238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.365798+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4239524987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.365981+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.366153+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/2502452747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.366328+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/3431139694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.366503+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/3831357174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.366665+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/2999465540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.366867+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1267840748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.508 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.367042+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/692561428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:07.367229+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:08.237140+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.388230+0000 mon.a [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.393567+0000 mon.a [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.394094+0000 mon.a [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.396639+0000 mon.a [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.397433+0000 mon.a [INF] from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.410262+0000 mon.a [INF] from='client.24707 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.410404+0000 mon.a [INF] from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.410705+0000 mon.a [INF] from='client.24713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.411827+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1607042823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.412087+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/3073094861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.412505+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/3119637976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.509 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.412773+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1282764311' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.412980+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/3710935882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.423947+0000 mon.a [INF] from='client.24767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.424492+0000 mon.a [INF] from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_reall api_service: [==========] Running 4 tests from 1 test suite. 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [----------] Global test environment set-up. 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [----------] 4 tests from LibRadosService 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5057 ms) 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ OK ] LibRadosService.RegisterLate (44 ms) 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: cluster: 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: id: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: health: HEALTH_OK 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.642 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: services: 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: data: 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pgs: 132 active+clean 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: io: 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T01:23:11.643 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: cluster: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: id: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: health: HEALTH_OK 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: services: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: data: 2024-02-18T01:23:11.644 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pgs: 132 active+clean 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: io: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: cluster: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: id: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: health: HEALTH_OK 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: services: 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-18T01:23:11.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: data: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pools: 6 pools, 132 pgs 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pgs: 132 active+clean 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: io: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:11.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: cluster: 2024-02-18T01:23:11.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: id: 358e970a-cdfa-11ee-95ba-87774f69a715 2024-02-18T01:23:11.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: health: HEALTH_WARN 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.357532+0000 mon.a (mon.0) 742 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.363886+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.105:0/799611238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.365798+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.105:0/4239524987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.365981+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.366153+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2502452747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.366328+0000 mon.c (mon.2) 40 : audit [INF] from='client.? v1:172.21.15.105:0/3431139694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.366503+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.105:0/3831357174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.366665+0000 mon.c (mon.2) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2999465540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.366867+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.105:0/1267840748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.367042+0000 mon.c (mon.2) 44 : audit [INF] from='client.? v1:172.21.15.105:0/692561428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.367229+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.506619+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? v1:172.21.15.105:0/1790282948' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:07.557468+0000 mgr.y (mgr.24475) 228 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:08.237140+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:08.415898+0000 mgr.y (mgr.24475) 229 : cluster [DBG] pgmap v174: 676 pgs: 150 creating+peering, 132 active+clean, 394 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:09.532648+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? v1:172.21.15.105:0/3374996541' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:10.379635+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.388230+0000 mon.a (mon.0) 744 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.393567+0000 mon.a (mon.0) 745 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.394094+0000 mon.a (mon.0) 746 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.396639+0000 mon.a (mon.0) 747 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.396846+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.397433+0000 mon.a (mon.0) 749 : audit [INF] from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.410262+0000 mon.a (mon.0) 750 : audit [INF] from='client.24707 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.410404+0000 mon.a (mon.0) 751 : audit [INF] from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.410705+0000 mon.a (mon.0) 752 : audit [INF] from='client.24713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:10.411585+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.411827+0000 mon.c (mon.2) 49 : audit [INF] from='client.? v1:172.21.15.105:0/1607042823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.412087+0000 mon.c (mon.2) 50 : audit [INF] from='client.? v1:172.21.15.105:0/3073094861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.412505+0000 mon.c (mon.2) 51 : audit [INF] from='client.? v1:172.21.15.105:0/3119637976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.412773+0000 mon.c (mon.2) 52 : audit [INF] from='client.? v1:172.21.15.105:0/1282764311' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.412980+0000 mon.c (mon.2) 53 : audit [INF] from='client.? v1:172.21.15.105:0/3710935882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:10.416780+0000 mgr.y (mgr.24475) 230 : cluster [DBG] pgmap v176: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:11.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.423947+0000 mon.a (mon.0) 754 : audit [INF] from='client.24767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.424492+0000 mon.a (mon.0) 755 : audit [INF] from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.425973+0000 mon.a (mon.0) 756 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.426536+0000 mon.a (mon.0) 757 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.434401+0000 mon.a (mon.0) 758 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.435570+0000 mon.a (mon.0) 759 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.436398+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.436643+0000 mon.a (mon.0) 760 : audit [INF] from='client.24785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.437558+0000 mon.a (mon.0) 761 : audit [INF] from='client.24791 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.437891+0000 mon.a (mon.0) 762 : audit [INF] from='client.24836 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.449929+0000 mon.a (mon.0) 763 : audit [INF] from='client.24827 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.450308+0000 mon.a (mon.0) 764 : audit [INF] from='client.24857 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.915950+0000 mon.a (mon.0) 765 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.916697+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:23:11.745 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.918324+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:10.921152+0000 mgr.y (mgr.24475) 231 : cluster [DBG] pgmap v177: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 786 B/s rd, 0 op/s 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: audit 2024-02-18T01:23:10.928065+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[19734]: cluster 2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.357532+0000 mon.a (mon.0) 742 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.363886+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.105:0/799611238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.365798+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.105:0/4239524987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.365981+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.366153+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2502452747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.366328+0000 mon.c (mon.2) 40 : audit [INF] from='client.? v1:172.21.15.105:0/3431139694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.366503+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.105:0/3831357174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.366665+0000 mon.c (mon.2) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2999465540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.366867+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.105:0/1267840748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.367042+0000 mon.c (mon.2) 44 : audit [INF] from='client.? v1:172.21.15.105:0/692561428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.367229+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.506619+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? v1:172.21.15.105:0/1790282948' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:07.557468+0000 mgr.y (mgr.24475) 228 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:08.237140+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:08.415898+0000 mgr.y (mgr.24475) 229 : cluster [DBG] pgmap v174: 676 pgs: 150 creating+peering, 132 active+clean, 394 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:09.532648+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? v1:172.21.15.105:0/3374996541' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:10.379635+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.388230+0000 mon.a (mon.0) 744 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.393567+0000 mon.a (mon.0) 745 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.394094+0000 mon.a (mon.0) 746 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.396639+0000 mon.a (mon.0) 747 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.396846+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.397433+0000 mon.a (mon.0) 749 : audit [INF] from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.410262+0000 mon.a (mon.0) 750 : audit [INF] from='client.24707 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.410404+0000 mon.a (mon.0) 751 : audit [INF] from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.410705+0000 mon.a (mon.0) 752 : audit [INF] from='client.24713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:10.411585+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.411827+0000 mon.c (mon.2) 49 : audit [INF] from='client.? v1:172.21.15.105:0/1607042823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.412087+0000 mon.c (mon.2) 50 : audit [INF] from='client.? v1:172.21.15.105:0/3073094861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.412505+0000 mon.c (mon.2) 51 : audit [INF] from='client.? v1:172.21.15.105:0/3119637976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.412773+0000 mon.c (mon.2) 52 : audit [INF] from='client.? v1:172.21.15.105:0/1282764311' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.412980+0000 mon.c (mon.2) 53 : audit [INF] from='client.? v1:172.21.15.105:0/3710935882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:10.416780+0000 mgr.y (mgr.24475) 230 : cluster [DBG] pgmap v176: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.423947+0000 mon.a (mon.0) 754 : audit [INF] from='client.24767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.748 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.424492+0000 mon.a (mon.0) 755 : audit [INF] from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.425973+0000 mon.a (mon.0) 756 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.426536+0000 mon.a (mon.0) 757 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.434401+0000 mon.a (mon.0) 758 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.435570+0000 mon.a (mon.0) 759 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.436398+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.436643+0000 mon.a (mon.0) 760 : audit [INF] from='client.24785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.437558+0000 mon.a (mon.0) 761 : audit [INF] from='client.24791 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.437891+0000 mon.a (mon.0) 762 : audit [INF] from='client.24836 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.449929+0000 mon.a (mon.0) 763 : audit [INF] from='client.24827 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.450308+0000 mon.a (mon.0) 764 : audit [INF] from='client.24857 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.915950+0000 mon.a (mon.0) 765 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.916697+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:23:11.749 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.918324+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:23:11.750 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:10.921152+0000 mgr.y (mgr.24475) 231 : cluster [DBG] pgmap v177: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 786 B/s rd, 0 op/s 2024-02-18T01:23:11.750 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: audit 2024-02-18T01:23:10.928065+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:11.750 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:11 smithi105 bash[27669]: cluster 2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:23:11.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.357532+0000 mon.a (mon.0) 742 : audit [INF] from='client.? v1:172.21.15.105:0/628642727' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi105-61649-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.363886+0000 mon.c (mon.2) 36 : audit [INF] from='client.? v1:172.21.15.105:0/799611238' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.365798+0000 mon.c (mon.2) 37 : audit [INF] from='client.? v1:172.21.15.105:0/4239524987' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.365981+0000 mon.c (mon.2) 38 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.366153+0000 mon.c (mon.2) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2502452747' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.366328+0000 mon.c (mon.2) 40 : audit [INF] from='client.? v1:172.21.15.105:0/3431139694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.366503+0000 mon.c (mon.2) 41 : audit [INF] from='client.? v1:172.21.15.105:0/3831357174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.366665+0000 mon.c (mon.2) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2999465540' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.366867+0000 mon.c (mon.2) 43 : audit [INF] from='client.? v1:172.21.15.105:0/1267840748' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.367042+0000 mon.c (mon.2) 44 : audit [INF] from='client.? v1:172.21.15.105:0/692561428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.367229+0000 mon.c (mon.2) 45 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.506619+0000 mon.c (mon.2) 46 : audit [DBG] from='client.? v1:172.21.15.105:0/1790282948' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:07.557468+0000 mgr.y (mgr.24475) 228 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:08.237140+0000 mon.c (mon.2) 47 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:08.415898+0000 mgr.y (mgr.24475) 229 : cluster [DBG] pgmap v174: 676 pgs: 150 creating+peering, 132 active+clean, 394 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:09.532648+0000 mon.c (mon.2) 48 : audit [DBG] from='client.? v1:172.21.15.105:0/3374996541' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:10.379635+0000 mon.a (mon.0) 743 : cluster [DBG] osdmap e67: 8 total, 8 up, 8 in 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.388230+0000 mon.a (mon.0) 744 : audit [INF] from='client.24716 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.393567+0000 mon.a (mon.0) 745 : audit [INF] from='client.24734 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.394094+0000 mon.a (mon.0) 746 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.396639+0000 mon.a (mon.0) 747 : audit [INF] from='client.24752 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.396846+0000 mon.a (mon.0) 748 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.397433+0000 mon.a (mon.0) 749 : audit [INF] from='client.24761 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.410262+0000 mon.a (mon.0) 750 : audit [INF] from='client.24707 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi105-61665-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.410404+0000 mon.a (mon.0) 751 : audit [INF] from='client.24832 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62008-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.410705+0000 mon.a (mon.0) 752 : audit [INF] from='client.24713 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi105-61672-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:10.411585+0000 mon.a (mon.0) 753 : cluster [DBG] osdmap e68: 8 total, 8 up, 8 in 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.411827+0000 mon.c (mon.2) 49 : audit [INF] from='client.? v1:172.21.15.105:0/1607042823' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.412087+0000 mon.c (mon.2) 50 : audit [INF] from='client.? v1:172.21.15.105:0/3073094861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.412505+0000 mon.c (mon.2) 51 : audit [INF] from='client.? v1:172.21.15.105:0/3119637976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.412773+0000 mon.c (mon.2) 52 : audit [INF] from='client.? v1:172.21.15.105:0/1282764311' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.412980+0000 mon.c (mon.2) 53 : audit [INF] from='client.? v1:172.21.15.105:0/3710935882' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:10.416780+0000 mgr.y (mgr.24475) 230 : cluster [DBG] pgmap v176: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.423947+0000 mon.a (mon.0) 754 : audit [INF] from='client.24767 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.424492+0000 mon.a (mon.0) 755 : audit [INF] from='client.24770 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.425973+0000 mon.a (mon.0) 756 : audit [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.426536+0000 mon.a (mon.0) 757 : audit [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.434401+0000 mon.a (mon.0) 758 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.435570+0000 mon.a (mon.0) 759 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.436398+0000 mon.c (mon.2) 54 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.436643+0000 mon.a (mon.0) 760 : audit [INF] from='client.24785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.437558+0000 mon.a (mon.0) 761 : audit [INF] from='client.24791 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.437891+0000 mon.a (mon.0) 762 : audit [INF] from='client.24836 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.449929+0000 mon.a (mon.0) 763 : audit [INF] from='client.24827 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.450308+0000 mon.a (mon.0) 764 : audit [INF] from='client.24857 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.915950+0000 mon.a (mon.0) 765 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.916697+0000 mon.a (mon.0) 766 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.918324+0000 mon.a (mon.0) 767 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:10.921152+0000 mgr.y (mgr.24475) 231 : cluster [DBG] pgmap v177: 900 pgs: 150 creating+peering, 132 active+clean, 618 unknown; 5.7 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 786 B/s rd, 0 op/s 2024-02-18T01:23:11.754 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: audit 2024-02-18T01:23:10.928065+0000 mon.a (mon.0) 768 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:11.755 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[20805]: cluster 2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api api_asio: [==========] Running 12 tests from 1 test suite. 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [----------] Global test environment set-up. 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [----------] 12 tests from AsioRados 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (2 ms) 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (3 ms) 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-02-18T01:23:11.788 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (3 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (3 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (3 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (2 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (0 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (2 ms) 2024-02-18T01:23:11.789 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (1 ms) 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (3 ms) 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [----------] 12 tests from AsioRados (23 ms total) 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [----------] Global test environment tear-down 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (11807 ms total) 2024-02-18T01:23:11.790 INFO:tasks.workunit.client.0.smithi105.stdout: api_asio: [ PASSED ] 12 tests. 2024-02-18T01:23:12.085 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] Global test environment set-up. 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 7 tests from LibRadosList 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.ListObjects (375 ms) 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (33 ms) 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo1,foo2,foo3 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo1 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo2 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo3 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo1,foo4,foo5 2024-02-18T01:23:12.086 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo4 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo5 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo1 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo6,foo7 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo7 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo6 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo4 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo5 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns2:foo7 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns2:foo6 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo1 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo1 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo2 2024-02-18T01:23:12.087 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo3 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (81 ms) 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 1 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 10 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 13 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 7 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 14 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 0 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 15 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 11 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 5 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 8 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 6 0 2024-02-18T01:23:12.088 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 3 0 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 4 0 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 12 0 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 9 0 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 2 0 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (73 ms) 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: x cursor=MIN 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=1 cursor=16:02547ec2:::1:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=10 cursor=16:52ea6a34:::10:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=13 cursor=16:566253c9:::13:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=7 cursor=16:5c6b0b28:::7:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=14 cursor=16:62a1935d:::14:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=0 cursor=16:6cac518f:::0:head 2024-02-18T01:23:12.089 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=15 cursor=16:863748b0:::15:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=11 cursor=16:89d3ae78:::11:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=5 cursor=16:b29083e3:::5:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=8 cursor=16:bd63b0f1:::8:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=6 cursor=16:c4fdafeb:::6:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=3 cursor=16:cfc208b3:::3:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=4 cursor=16:d83876eb:::4:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=12 cursor=16:de5d7c5f:::12:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=9 cursor=16:e960b815:::9:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > oid=2 cursor=16:f905c69b:::2:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: FIRST> seek to MIN oid=1 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=1 cursor=16:02547ec2:::1:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:02547ec2:::1:head 2024-02-18T01:23:12.090 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:02547ec2:::1:head -> 1 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=10 cursor=16:52ea6a34:::10:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:52ea6a34:::10:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:52ea6a34:::10:head -> 10 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=13 cursor=16:566253c9:::13:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:566253c9:::13:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:566253c9:::13:head -> 13 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=7 cursor=16:5c6b0b28:::7:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:5c6b0b28:::7:head 2024-02-18T01:23:12.091 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:5c6b0b28:::7:head -> 7 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=14 cursor=16:62a1935d:::14:head 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:62a1935d:::14:head 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:62a1935d:::14:head -> 14 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=0 cursor=16:6cac518f:::0:head 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:6cac518f:::0:head 2024-02-18T01:23:12.143 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:6cac518f:::0:head -> 0 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=15 cursor=16:863748b0:::15:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:863748b0:::15:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:863748b0:::15:head -> 15 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=11 cursor=16:89d3ae78:::11:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:89d3ae78:::11:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:89d3ae78:::11:head -> 11 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=5 cursor=16:b29083e3:::5:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:b29083e3:::5:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:b29083e3:::5:head -> 5 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=8 cursor=16:bd63b0f1:::8:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:bd63b0f1:::8:head 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:bd63b0f1:::8:head -> 8 2024-02-18T01:23:12.144 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=6 cursor=16:c4fdafeb:::6:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:c4fdafeb:::6:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:c4fdafeb:::6:head -> 6 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=3 cursor=16:cfc208b3:::3:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:cfc208b3:::3:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:cfc208b3:::3:head -> 3 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=4 cursor=16:d83876eb:::4:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:d83876eb:::4:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:d83876eb:::4:head -> 4 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=12 cursor=16:de5d7c5f:::12:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:de5d7c5f:::12:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:de5d7c5f:::12:head -> 12 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=9 cursor=16:e960b815:::9:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:e960b815:::9:head 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:e960b815:::9:head -> 9 2024-02-18T01:23:12.145 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : oid=2 cursor=16:f905c69b:::2:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:f905c69b:::2:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:f905c69b:::2:head -> 2 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:6cac518f:::0:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:6cac518f:::0:head expected=16:6cac518f:::0:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:6cac518f:::0:head -> 0 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=0 expected=0 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:f905c69b:::2:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:f905c69b:::2:head expected=16:f905c69b:::2:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:f905c69b:::2:head -> 2 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=2 expected=2 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:e960b815:::9:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:e960b815:::9:head expected=16:e960b815:::9:head 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:e960b815:::9:head -> 9 2024-02-18T01:23:12.146 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=9 expected=9 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:de5d7c5f:::12:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:de5d7c5f:::12:head expected=16:de5d7c5f:::12:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:de5d7c5f:::12:head -> 12 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=12 expected=12 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:d83876eb:::4:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:d83876eb:::4:head expected=16:d83876eb:::4:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:d83876eb:::4:head -> 4 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=4 expected=4 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:cfc208b3:::3:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:cfc208b3:::3:head expected=16:cfc208b3:::3:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:cfc208b3:::3:head -> 3 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=3 expected=3 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:c4fdafeb:::6:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:c4fdafeb:::6:head expected=16:c4fdafeb:::6:head 2024-02-18T01:23:12.147 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:c4fdafeb:::6:head -> 6 2024-02-18T01:23:12.148 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=6 expected=6 2024-02-18T01:23:12.148 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:bd63b0f1:::8:head 2024-02-18T01:23:12.148 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:bd63b0f1:::8:head expected=16:bd63b0f1:::8:head 2024-02-18T01:23:12.148 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:bd63b0f1:::8:head -> 8 2024-02-18T01:23:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=8 expecte list_parallel: process_1_[62217]: starting. 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: creating pool ceph_test_rados_list_parallel.smithi105-62195 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: created object 0... 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: created object 25... 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: created object 49... 2024-02-18T01:23:12.295 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: finishing. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_1_[62217]: shutting down. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: starting. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: listing objects. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: listed object 0... 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: listed object 25... 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: saw 50 objects 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_2_[62218]: shutting down. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: starting. 2024-02-18T01:23:12.296 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: creating pool ceph_test_rados_list_parallel.smithi105-62195 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: created object 0... 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: created object 25... 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: created object 49... 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: finishing. 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_3_[63003]: shutting down. 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: starting. 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: listing objects. 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: listed object 0... 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: listed object 25... 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: saw 43 objects 2024-02-18T01:23:12.297 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_4_[63004]: shutting down. 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: starting. 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: removed 25 objects... 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: removed half of the objects 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: removed 50 objects... 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: removed 50 objects 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_5_[63005]: shutting down. 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: starting. 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: creating pool ceph_test_rados_list_parallel.smithi105-62195 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: created object 0... 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: created object 25... 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: created object 49... 2024-02-18T01:23:12.298 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: finishing. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_6_[63053]: shutting down. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: starting. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: listing objects. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: listed object 0... 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: listed object 25... 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: listed object 50... 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: saw 54 objects 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_7_[63054]: shutting down. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: starting. 2024-02-18T01:23:12.299 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: added 25 objects... 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: added half of the objects 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: added 50 objects... 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: added 50 objects 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_8_[63055]: shutting down. 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.300 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.492 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.492 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: starting. 2024-02-18T01:23:12.492 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: creating pool ceph_test_rados_list_parallel.smithi105-62195 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: created object 0... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: created object 25... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: created object 49... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: finishing. 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_9_[63101]: shutting down. 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: starting. 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: listing objects. 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: listed object 0... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: listed object 25... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: listed object 50... 2024-02-18T01:23:12.493 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: listed object 75... 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: saw 98 objects 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_10_[63102]: shutting down. 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: starting. 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: removed 25 objects... 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: removed half of the objects 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: removed 50 objects... 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: removed 50 objects 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_13_[63105]: shutting down. 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.494 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: starting. 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: added 25 objects... 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: added half of the objects 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: added 50 objects... 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: added 50 objects 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_11_[63103]: shutting down. 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: starting. 2024-02-18T01:23:12.495 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: added 25 objects... 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: added half of the objects 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: added 50 objects... 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: added 50 objects 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_12_[63104]: shutting down. 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: starting. 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: creating pool ceph_test_rados_list_parallel.smithi105-62195 2024-02-18T01:23:12.496 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: created object 0... 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: created object 25... 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: created object 49... 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: finishing. 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_14_[63181]: shutting down. 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: starting. 2024-02-18T01:23:12.497 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listing objects. 2024-02-18T01:23:12.498 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 0... 2024-02-18T01:23:12.974 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 2y_mean_it": true}]: dispatch 2024-02-18T01:23:12.974 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.425973+0000 mon.a [INF] from='client.24782 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.974 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.426536+0000 mon.a [INF] from='client.24728 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.974 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.434401+0000 mon.a [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.974 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.435570+0000 mon.a [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.436398+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.436643+0000 mon.a [INF] from='client.24785 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.437558+0000 mon.a [INF] from='client.24791 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.437891+0000 mon.a [INF] from='client.24836 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.449929+0000 mon.a [INF] from='client.24827 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.450308+0000 mon.a [INF] from='client.24857 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.915950+0000 mon.a [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.918324+0000 mon.a [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:10.928065+0000 mon.a [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.389015+0000 mon.a [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.417168+0000 mon.a [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.975 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.417407+0000 mon.a [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.417871+0000 mon.a [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.418186+0000 mon.a [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.418705+0000 mon.a [INF] from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.419334+0000 mon.a [INF] from='client.24767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.419818+0000 mon.a [INF] from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.420084+0000 mon.a [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.420627+0000 mon.a [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.420847+0000 mon.a [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.421678+0000 mon.a [INF] from='client.24785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.422088+0000 mon.a [INF] from='client.24791 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.422328+0000 mon.a [INF] from='client.24836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.422534+0000 mon.a [INF] from='client.24827 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.422707+0000 mon.a [INF] from='client.24857 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:12.976 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.422878+0000 mon.a [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:12.977 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.429941+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.417168+0000 mon.a (mon.0) 770 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.417407+0000 mon.a (mon.0) 771 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.417871+0000 mon.a (mon.0) 772 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.418186+0000 mon.a (mon.0) 773 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.418705+0000 mon.a (mon.0) 774 : audit [INF] from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.419334+0000 mon.a (mon.0) 775 : audit [INF] from='client.24767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.419818+0000 mon.a (mon.0) 776 : audit [INF] from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.420084+0000 mon.a (mon.0) 777 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.420627+0000 mon.a (mon.0) 778 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.420847+0000 mon.a (mon.0) 779 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.421678+0000 mon.a (mon.0) 780 : audit [INF] from='client.24785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.422088+0000 mon.a (mon.0) 781 : audit [INF] from='client.24791 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.422328+0000 mon.a (mon.0) 782 : audit [INF] from='client.24836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.417168+0000 mon.a (mon.0) 770 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.417407+0000 mon.a (mon.0) 771 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.417871+0000 mon.a (mon.0) 772 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.418186+0000 mon.a (mon.0) 773 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.418705+0000 mon.a (mon.0) 774 : audit [INF] from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.419334+0000 mon.a (mon.0) 775 : audit [INF] from='client.24767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.419818+0000 mon.a (mon.0) 776 : audit [INF] from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.420084+0000 mon.a (mon.0) 777 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.420627+0000 mon.a (mon.0) 778 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.420847+0000 mon.a (mon.0) 779 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.421678+0000 mon.a (mon.0) 780 : audit [INF] from='client.24785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.422088+0000 mon.a (mon.0) 781 : audit [INF] from='client.24791 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.422328+0000 mon.a (mon.0) 782 : audit [INF] from='client.24836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.422534+0000 mon.a (mon.0) 783 : audit [INF] from='client.24827 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.422707+0000 mon.a (mon.0) 784 : audit [INF] from='client.24857 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.422878+0000 mon.a (mon.0) 785 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: cluster 2024-02-18T01:23:11.423117+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.429941+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.441679+0000 mon.a (mon.0) 787 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.490888+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.491488+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.491824+0000 mon.a (mon.0) 788 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.492707+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: cluster 2024-02-18T01:23:11.493029+0000 client.admin (client.25051) 0 : cluster [INF] onexx 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.493482+0000 mon.a (mon.0) 789 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.493775+0000 mon.a (mon.0) 790 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.502585+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.505596+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[27669]: audit 2024-02-18T01:23:11.630552+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? v1:172.21.15.105:0/1825294924' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:13.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.422534+0000 mon.a (mon.0) 783 : audit [INF] from='client.24827 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.422707+0000 mon.a (mon.0) 784 : audit [INF] from='client.24857 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.422878+0000 mon.a (mon.0) 785 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: cluster 2024-02-18T01:23:11.423117+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.429941+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.441679+0000 mon.a (mon.0) 787 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.490888+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.491488+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.491824+0000 mon.a (mon.0) 788 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.492707+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: cluster 2024-02-18T01:23:11.493029+0000 client.admin (client.25051) 0 : cluster [INF] onexx 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.493482+0000 mon.a (mon.0) 789 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.493775+0000 mon.a (mon.0) 790 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.502585+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.505596+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:12 smithi105 bash[19734]: audit 2024-02-18T01:23:11.630552+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? v1:172.21.15.105:0/1825294924' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.417168+0000 mon.a (mon.0) 770 : audit [INF] from='client.24716 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi105-61657-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.417407+0000 mon.a (mon.0) 771 : audit [INF] from='client.24734 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi105-61697-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.417871+0000 mon.a (mon.0) 772 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.418186+0000 mon.a (mon.0) 773 : audit [INF] from='client.24752 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.418705+0000 mon.a (mon.0) 774 : audit [INF] from='client.24761 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi105-61891-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.419334+0000 mon.a (mon.0) 775 : audit [INF] from='client.24767 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi105-61833-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.419818+0000 mon.a (mon.0) 776 : audit [INF] from='client.24770 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi105-61914-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.420084+0000 mon.a (mon.0) 777 : audit [INF] from='client.24782 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi105-61952-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.420627+0000 mon.a (mon.0) 778 : audit [INF] from='client.24728 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi105-61709-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.420847+0000 mon.a (mon.0) 779 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi105-61689-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.421678+0000 mon.a (mon.0) 780 : audit [INF] from='client.24785 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi105-61957-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.422088+0000 mon.a (mon.0) 781 : audit [INF] from='client.24791 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi105-61946-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.422328+0000 mon.a (mon.0) 782 : audit [INF] from='client.24836 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi105-62148-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.422534+0000 mon.a (mon.0) 783 : audit [INF] from='client.24827 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.422707+0000 mon.a (mon.0) 784 : audit [INF] from='client.24857 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62048-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.422878+0000 mon.a (mon.0) 785 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: cluster 2024-02-18T01:23:11.423117+0000 mon.a (mon.0) 786 : cluster [DBG] osdmap e69: 8 total, 8 up, 8 in 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.429941+0000 mon.c (mon.2) 55 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.441679+0000 mon.a (mon.0) 787 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.490888+0000 mon.b (mon.1) 28 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.491488+0000 mon.b (mon.1) 29 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.491824+0000 mon.a (mon.0) 788 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.492707+0000 mon.b (mon.1) 30 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: cluster 2024-02-18T01:23:11.493029+0000 client.admin (client.25051) 0 : cluster [INF] onexx 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.493482+0000 mon.a (mon.0) 789 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.493775+0000 mon.a (mon.0) 790 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.502585+0000 mon.b (mon.1) 31 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.505596+0000 mon.b (mon.1) 32 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:12 smithi155 bash[20805]: audit 2024-02-18T01:23:11.630552+0000 mon.b (mon.1) 33 : audit [DBG] from='client.? v1:172.21.15.105:0/1825294924' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-02-18T01:23:13.926 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01 api_c_read_operations: Running main() from gmock_main.cc 2024-02-18T01:23:13.926 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (205 ms) 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (110 ms) 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (14 ms) 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (23 ms) 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-02-18T01:23:13.927 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (7 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (8 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (5 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (3 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-02-18T01:23:13.928 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (11 ms) 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (13 ms) 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (11 ms) 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (7 ms) 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (433 ms total) 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-02-18T01:23:13.929 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (13767 ms total) 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-02-18T01:23:13.930 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (39 ms) 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (51 ms) 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (13718 ms) 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (13808 ms total) 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (13808 ms total) 2024-02-18T01:23:13.931 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-02-18T01:23:14.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.269476+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.501411+0000 mon.a (mon.0) 791 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.501927+0000 mon.a (mon.0) 792 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: cluster 2024-02-18T01:23:12.921887+0000 mgr.y (mgr.24475) 232 : cluster [DBG] pgmap v180: 772 pgs: 32 creating+peering, 676 active+clean, 64 unknown; 32 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 95 KiB/s wr, 266 op/s 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: cluster 2024-02-18T01:23:12.963831+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.964650+0000 mon.c (mon.2) 57 : audit [INF] from='client.? v1:172.21.15.105:0/1933155993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.965202+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.965317+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.973506+0000 mon.a (mon.0) 794 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:12.978209+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: cluster 2024-02-18T01:23:12.978416+0000 client.admin (client.25051) 0 : cluster [INF] twoxx 2024-02-18T01:23:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.013541+0000 mon.a (mon.0) 795 : audit [INF] from='client.25031 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.270863+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: cluster 2024-02-18T01:23:13.902587+0000 mon.a (mon.0) 796 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.910780+0000 mon.a (mon.0) 797 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.911218+0000 mon.a (mon.0) 798 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.911512+0000 mon.a (mon.0) 799 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.917394+0000 mon.a (mon.0) 800 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.918016+0000 mon.a (mon.0) 801 : audit [INF] from='client.25031 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: cluster 2024-02-18T01:23:13.919376+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.923240+0000 mon.c (mon.2) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1085894363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[27669]: audit 2024-02-18T01:23:13.923427+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.269476+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.501411+0000 mon.a (mon.0) 791 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.501927+0000 mon.a (mon.0) 792 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: cluster 2024-02-18T01:23:12.921887+0000 mgr.y (mgr.24475) 232 : cluster [DBG] pgmap v180: 772 pgs: 32 creating+peering, 676 active+clean, 64 unknown; 32 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 95 KiB/s wr, 266 op/s 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: cluster 2024-02-18T01:23:12.963831+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.964650+0000 mon.c (mon.2) 57 : audit [INF] from='client.? v1:172.21.15.105:0/1933155993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.965202+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.965317+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.973506+0000 mon.a (mon.0) 794 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:12.978209+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: cluster 2024-02-18T01:23:12.978416+0000 client.admin (client.25051) 0 : cluster [INF] twoxx 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.013541+0000 mon.a (mon.0) 795 : audit [INF] from='client.25031 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.270863+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: cluster 2024-02-18T01:23:13.902587+0000 mon.a (mon.0) 796 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.910780+0000 mon.a (mon.0) 797 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.911218+0000 mon.a (mon.0) 798 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.911512+0000 mon.a (mon.0) 799 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.917394+0000 mon.a (mon.0) 800 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.918016+0000 mon.a (mon.0) 801 : audit [INF] from='client.25031 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: cluster 2024-02-18T01:23:13.919376+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.923240+0000 mon.c (mon.2) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1085894363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:14 smithi105 bash[19734]: audit 2024-02-18T01:23:13.923427+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.269476+0000 mon.c (mon.2) 56 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.501411+0000 mon.a (mon.0) 791 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.501927+0000 mon.a (mon.0) 792 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: cluster 2024-02-18T01:23:12.921887+0000 mgr.y (mgr.24475) 232 : cluster [DBG] pgmap v180: 772 pgs: 32 creating+peering, 676 active+clean, 64 unknown; 32 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 95 KiB/s wr, 266 op/s 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: cluster 2024-02-18T01:23:12.963831+0000 mon.a (mon.0) 793 : cluster [DBG] osdmap e70: 8 total, 8 up, 8 in 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.964650+0000 mon.c (mon.2) 57 : audit [INF] from='client.? v1:172.21.15.105:0/1933155993' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.965202+0000 mon.b (mon.1) 34 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.965317+0000 mon.b (mon.1) 35 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.973506+0000 mon.a (mon.0) 794 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:12.978209+0000 mon.b (mon.1) 36 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: cluster 2024-02-18T01:23:12.978416+0000 client.admin (client.25051) 0 : cluster [INF] twoxx 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.013541+0000 mon.a (mon.0) 795 : audit [INF] from='client.25031 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.270863+0000 mon.c (mon.2) 58 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: cluster 2024-02-18T01:23:13.902587+0000 mon.a (mon.0) 796 : cluster [WRN] Health check failed: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.910780+0000 mon.a (mon.0) 797 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.911218+0000 mon.a (mon.0) 798 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.911512+0000 mon.a (mon.0) 799 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.917394+0000 mon.a (mon.0) 800 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.918016+0000 mon.a (mon.0) 801 : audit [INF] from='client.25031 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi105-61649-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: cluster 2024-02-18T01:23:13.919376+0000 mon.a (mon.0) 802 : cluster [DBG] osdmap e71: 8 total, 8 up, 8 in 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.923240+0000 mon.c (mon.2) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1085894363' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:14 smithi155 bash[20805]: audit 2024-02-18T01:23:13.923427+0000 mon.b (mon.1) 37 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:14.935 INFO:tasks.workunit.client.0.smithi105.stdout:5... 2024-02-18T01:23:14.935 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 50... 2024-02-18T01:23:14.935 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 75... 2024-02-18T01:23:14.935 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 100... 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: listed object 125... 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: saw 150 objects 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_15_[63182]: shutting down. 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: starting. 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: added 25 objects... 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: added half of the objects 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: added 50 objects... 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: added 50 objects 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: process_16_[63183]: shutting down. 2024-02-18T01:23:14.936 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.937 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.937 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.937 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.937 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******************************* 2024-02-18T01:23:14.937 INFO:tasks.workunit.client.0.smithi105.stdout: list_parallel: ******* SUCCESS ********** 2024-02-18T01:23:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:13.995084+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.018804+0000 mon.c (mon.2) 60 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.272350+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.403343+0000 mon.a (mon.0) 803 : audit [INF] from='client.25145 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.410330+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.410564+0000 mgr.y (mgr.24475) 233 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.410925+0000 mon.a (mon.0) 804 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.411359+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.411519+0000 mgr.y (mgr.24475) 234 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.412136+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.412305+0000 mgr.y (mgr.24475) 235 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.412784+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.412949+0000 mgr.y (mgr.24475) 236 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.413069+0000 mon.a (mon.0) 805 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.413440+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.413576+0000 mgr.y (mgr.24475) 237 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:13.995084+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.018804+0000 mon.c (mon.2) 60 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.272350+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.403343+0000 mon.a (mon.0) 803 : audit [INF] from='client.25145 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.410330+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.410564+0000 mgr.y (mgr.24475) 233 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.410925+0000 mon.a (mon.0) 804 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.411359+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.411519+0000 mgr.y (mgr.24475) 234 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.412136+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.412305+0000 mgr.y (mgr.24475) 235 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.412784+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.412949+0000 mgr.y (mgr.24475) 236 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.413069+0000 mon.a (mon.0) 805 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.413440+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.413576+0000 mgr.y (mgr.24475) 237 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.414062+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.414209+0000 mgr.y (mgr.24475) 238 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.414651+0000 mon.c (mon.2) 68 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.414786+0000 mgr.y (mgr.24475) 239 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.415321+0000 mon.c (mon.2) 69 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.415343+0000 mon.a (mon.0) 806 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.415473+0000 mgr.y (mgr.24475) 240 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.416022+0000 mon.c (mon.2) 70 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.416160+0000 mgr.y (mgr.24475) 241 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.416330+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.416583+0000 mon.c (mon.2) 71 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.416732+0000 mgr.y (mgr.24475) 242 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.453386+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.456867+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.831016+0000 mon.a (mon.0) 807 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.831825+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.832597+0000 mon.a (mon.0) 808 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.414062+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.414209+0000 mgr.y (mgr.24475) 238 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.414651+0000 mon.c (mon.2) 68 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.414786+0000 mgr.y (mgr.24475) 239 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.415321+0000 mon.c (mon.2) 69 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.415343+0000 mon.a (mon.0) 806 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.415473+0000 mgr.y (mgr.24475) 240 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.416022+0000 mon.c (mon.2) 70 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.416160+0000 mgr.y (mgr.24475) 241 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.416330+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.416583+0000 mon.c (mon.2) 71 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.416732+0000 mgr.y (mgr.24475) 242 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.453386+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.456867+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:16.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.831016+0000 mon.a (mon.0) 807 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.831825+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.832597+0000 mon.a (mon.0) 808 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.833093+0000 mon.a (mon.0) 809 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.833734+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.835730+0000 mon.a (mon.0) 810 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.836136+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.836380+0000 mon.a (mon.0) 811 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.923937+0000 mgr.y (mgr.24475) 243 : cluster [DBG] pgmap v183: 556 pgs: 356 active+clean, 200 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 173 op/s 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.926275+0000 mon.a (mon.0) 812 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.926395+0000 mon.a (mon.0) 813 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.247 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.926576+0000 mon.a (mon.0) 814 : audit [INF] from='client.25145 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.926851+0000 mon.a (mon.0) 815 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.927163+0000 mon.a (mon.0) 816 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.927257+0000 mon.a (mon.0) 817 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.927394+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.930070+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.833093+0000 mon.a (mon.0) 809 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.833734+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.248 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.835730+0000 mon.a (mon.0) 810 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.836136+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.836380+0000 mon.a (mon.0) 811 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.923937+0000 mgr.y (mgr.24475) 243 : cluster [DBG] pgmap v183: 556 pgs: 356 active+clean, 200 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 173 op/s 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.926275+0000 mon.a (mon.0) 812 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.926395+0000 mon.a (mon.0) 813 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.926576+0000 mon.a (mon.0) 814 : audit [INF] from='client.25145 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.926851+0000 mon.a (mon.0) 815 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.927163+0000 mon.a (mon.0) 816 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.927257+0000 mon.a (mon.0) 817 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.927394+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.930070+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.930668+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.249 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.931319+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.931608+0000 mon.c (mon.2) 74 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.931947+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.933322+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.105:0/280023018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.936249+0000 mon.a (mon.0) 819 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.936887+0000 mon.a (mon.0) 820 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.937409+0000 mon.a (mon.0) 821 : audit [INF] from='client.25160 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.937702+0000 mon.a (mon.0) 822 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.938095+0000 mon.a (mon.0) 823 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.938639+0000 mon.a (mon.0) 824 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:14.977695+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.250 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.930668+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.931319+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.931608+0000 mon.c (mon.2) 74 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.931947+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.933322+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.105:0/280023018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.936249+0000 mon.a (mon.0) 819 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.936887+0000 mon.a (mon.0) 820 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.937409+0000 mon.a (mon.0) 821 : audit [INF] from='client.25160 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.937702+0000 mon.a (mon.0) 822 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.938095+0000 mon.a (mon.0) 823 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.938639+0000 mon.a (mon.0) 824 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.251 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:14.977695+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:15.016551+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:15.273663+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[27669]: audit 2024-02-18T01:23:15.401088+0000 mon.a (mon.0) 825 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:15.016551+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:15.273663+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.252 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:15 smithi105 bash[19734]: audit 2024-02-18T01:23:15.401088+0000 mon.a (mon.0) 825 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:13.995084+0000 mon.b (mon.1) 38 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.018804+0000 mon.c (mon.2) 60 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.272350+0000 mon.c (mon.2) 61 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.403343+0000 mon.a (mon.0) 803 : audit [INF] from='client.25145 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.410330+0000 mon.c (mon.2) 62 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.253 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.410564+0000 mgr.y (mgr.24475) 233 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.410925+0000 mon.a (mon.0) 804 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.411359+0000 mon.c (mon.2) 63 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.411519+0000 mgr.y (mgr.24475) 234 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.412136+0000 mon.c (mon.2) 64 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.412305+0000 mgr.y (mgr.24475) 235 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.412784+0000 mon.c (mon.2) 65 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.412949+0000 mgr.y (mgr.24475) 236 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.413069+0000 mon.a (mon.0) 805 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.413440+0000 mon.c (mon.2) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.413576+0000 mgr.y (mgr.24475) 237 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.414062+0000 mon.c (mon.2) 67 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.414209+0000 mgr.y (mgr.24475) 238 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.254 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.414651+0000 mon.c (mon.2) 68 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.414786+0000 mgr.y (mgr.24475) 239 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.415321+0000 mon.c (mon.2) 69 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.255 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.415343+0000 mon.a (mon.0) 806 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.415473+0000 mgr.y (mgr.24475) 240 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.416022+0000 mon.c (mon.2) 70 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.416160+0000 mgr.y (mgr.24475) 241 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.416330+0000 mon.b (mon.1) 39 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.416583+0000 mon.c (mon.2) 71 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.416732+0000 mgr.y (mgr.24475) 242 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.453386+0000 osd.2 (osd.2) 3 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.456867+0000 osd.2 (osd.2) 4 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.831016+0000 mon.a (mon.0) 807 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.831825+0000 mon.b (mon.1) 40 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.832597+0000 mon.a (mon.0) 808 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.833093+0000 mon.a (mon.0) 809 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.256 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.833734+0000 mon.c (mon.2) 72 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.835730+0000 mon.a (mon.0) 810 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.836136+0000 mon.c (mon.2) 73 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.836380+0000 mon.a (mon.0) 811 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.923937+0000 mgr.y (mgr.24475) 243 : cluster [DBG] pgmap v183: 556 pgs: 356 active+clean, 200 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 22 KiB/s wr, 173 op/s 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.926275+0000 mon.a (mon.0) 812 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.926395+0000 mon.a (mon.0) 813 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.926576+0000 mon.a (mon.0) 814 : audit [INF] from='client.25145 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.926851+0000 mon.a (mon.0) 815 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.927163+0000 mon.a (mon.0) 816 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.927257+0000 mon.a (mon.0) 817 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.927394+0000 mon.a (mon.0) 818 : cluster [DBG] osdmap e72: 8 total, 8 up, 8 in 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.930070+0000 mon.b (mon.1) 41 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.930668+0000 mon.b (mon.1) 42 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.257 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.931319+0000 mon.b (mon.1) 43 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.931608+0000 mon.c (mon.2) 74 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.931947+0000 mon.b (mon.1) 44 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.933322+0000 mon.c (mon.2) 75 : audit [INF] from='client.? v1:172.21.15.105:0/280023018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.936249+0000 mon.a (mon.0) 819 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.936887+0000 mon.a (mon.0) 820 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.937409+0000 mon.a (mon.0) 821 : audit [INF] from='client.25160 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.937702+0000 mon.a (mon.0) 822 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.938095+0000 mon.a (mon.0) 823 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.938639+0000 mon.a (mon.0) 824 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:14.977695+0000 mon.c (mon.2) 76 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:15.016551+0000 mon.b (mon.1) 45 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:15.273663+0000 mon.c (mon.2) 77 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:16.258 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:15 smithi155 bash[20805]: audit 2024-02-18T01:23:15.401088+0000 mon.a (mon.0) 825 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:16.414 INFO:tasks.workunit.client.0.smithi105.stdout::23:11.441679+0000 mon.a [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi105-61976-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:16.414 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.490888+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.414 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.491488+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.491824+0000 mon.a [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.492707+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.493029+0000 client.admin [INF] onexx 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.493482+0000 mon.a [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.493775+0000 mon.a [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.502585+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:11.505596+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:13.995084+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.018804+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.403343+0000 mon.a [INF] from='client.25145 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.410330+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.410925+0000 mon.a [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-02-18T01:23:16.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.411359+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.412136+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.412784+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.413069+0000 mon.a [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.413440+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.414062+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.414651+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.415321+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.415343+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.416022+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.416330+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.416583+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.831016+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.831825+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.832597+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.833093+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.833734+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.835730+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.836136+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.836380+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.926275+0000 mon.a [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.926395+0000 mon.a [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi105-61665-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.926576+0000 mon.a [INF] from='client.25145 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.926851+0000 mon.a [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:16.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.927163+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi105-61952-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.927257+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.930070+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.930668+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.931319+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.931608+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.931947+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.933322+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/280023018' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:16.418 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.936249+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:16] "GET /metrics HTTP/1.1" 200 184464 "" "Prometheus/2.33.4" 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.936887+0000 mon.a [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"er open_pools_parallel: process_1_[62282]: starting. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: creating pool ceph_test_rados_open_pools_parallel.smithi105-62267 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: created object 0... 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: created object 25... 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: created object 49... 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: finishing. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_1_[62282]: shutting down. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_2_[62283]: starting. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_2_[62283]: rados_pool_create. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_2_[62283]: rados_ioctx_create. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_2_[62283]: shutting down. 2024-02-18T01:23:17.955 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: ******************************* 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: starting. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: creating pool ceph_test_rados_open_pools_parallel.smithi105-62267 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: created object 0... 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: created object 25... 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: created object 49... 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: finishing. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_3_[62849]: shutting down. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: ******************************* 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_4_[62850]: starting. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_4_[62850]: rados_pool_create. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_4_[62850]: rados_ioctx_create. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: process_4_[62850]: shutting down. 2024-02-18T01:23:17.956 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: ******************************* 2024-02-18T01:23:17.957 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: ******************************* 2024-02-18T01:23:17.957 INFO:tasks.workunit.client.0.smithi105.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-02-18T01:23:17.957 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.735732+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: cluster 2024-02-18T01:23:14.740302+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:17.957 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.401787+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: cluster 2024-02-18T01:23:15.490303+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: cluster 2024-02-18T01:23:15.494346+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:17.958 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.853312+0000 mon.a (mon.0) 826 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.735732+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: cluster 2024-02-18T01:23:14.740302+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.401787+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: cluster 2024-02-18T01:23:15.490303+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: cluster 2024-02-18T01:23:15.494346+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.853312+0000 mon.a (mon.0) 826 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.853980+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.854220+0000 mon.a (mon.0) 827 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.854775+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:15.855178+0000 mon.a (mon.0) 828 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:16.275294+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[19734]: audit 2024-02-18T01:23:16.279077+0000 mon.a (mon.0) 829 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.735732+0000 osd.3 (osd.3) 3 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: cluster 2024-02-18T01:23:14.740302+0000 osd.3 (osd.3) 4 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.401787+0000 mon.c (mon.2) 78 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: cluster 2024-02-18T01:23:15.490303+0000 osd.1 (osd.1) 3 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: cluster 2024-02-18T01:23:15.494346+0000 osd.1 (osd.1) 4 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.853312+0000 mon.a (mon.0) 826 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.853980+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.854220+0000 mon.a (mon.0) 827 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.854775+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:15.855178+0000 mon.a (mon.0) 828 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:16.275294+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:18.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:17 smithi105 bash[27669]: audit 2024-02-18T01:23:16.279077+0000 mon.a (mon.0) 829 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.853980+0000 mon.b (mon.1) 46 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.854220+0000 mon.a (mon.0) 827 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.854775+0000 mon.c (mon.2) 79 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:15.855178+0000 mon.a (mon.0) 828 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:16.275294+0000 mon.c (mon.2) 80 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:17 smithi155 bash[20805]: audit 2024-02-18T01:23:16.279077+0000 mon.a (mon.0) 829 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:19.340 INFO:tasks.workunit.client.0.smithi105.stdout:asure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.937409+0000 mon.a [INF] from='client.25160 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.937702+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.938095+0000 mon.a [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.938639+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:14.977695+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.016551+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.401088+0000 mon.a [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.401787+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.853312+0000 mon.a [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.853980+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:19.341 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.854220+0000 mon.a [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.854775+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:15.855178+0000 mon.a [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:16.279077+0000 mon.a [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:16.413116+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:16.413467+0000 client.admin [INF] threexx 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.118152+0000 mon.a [INF] from='client.25160 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.118369+0000 mon.a [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.934702+0000 mon.a [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.935561+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.936498+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.937398+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.938073+0000 mon.c [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:17.938483+0000 mon.b [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.342 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.018613+0000 mon.a [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:19.343 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.018682+0000 mon.a [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:19.343 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.018751+0000 mon.a [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:19.343 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.018812+0000 mon.a [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:19.343 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.449287+0000 mon.a [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:15.704219+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:15.805250+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:16.413116+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:16.413467+0000 client.admin (client.25051) 0 : cluster [INF] threexx 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:16.503961+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:16.508410+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:16.925140+0000 mgr.y (mgr.24475) 244 : cluster [DBG] pgmap v185: 548 pgs: 356 active+clean, 192 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.118152+0000 mon.a (mon.0) 830 : audit [INF] from='client.25160 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.118369+0000 mon.a (mon.0) 831 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.276839+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.569100+0000 mgr.y (mgr.24475) 245 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.934702+0000 mon.a (mon.0) 832 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.935561+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.936498+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:17.937239+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.937398+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.938073+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:15.704219+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:15.805250+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:16.413116+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:16.413467+0000 client.admin (client.25051) 0 : cluster [INF] threexx 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:16.503961+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:16.508410+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:16.925140+0000 mgr.y (mgr.24475) 244 : cluster [DBG] pgmap v185: 548 pgs: 356 active+clean, 192 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.118152+0000 mon.a (mon.0) 830 : audit [INF] from='client.25160 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.118369+0000 mon.a (mon.0) 831 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.276839+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.569100+0000 mgr.y (mgr.24475) 245 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.934702+0000 mon.a (mon.0) 832 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.935561+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.936498+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:17.937239+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.937398+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.938073+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:17.938483+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.018613+0000 mon.a (mon.0) 834 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.018682+0000 mon.a (mon.0) 835 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.018751+0000 mon.a (mon.0) 836 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.018812+0000 mon.a (mon.0) 837 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: cluster 2024-02-18T01:23:18.019156+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.278450+0000 mon.c (mon.2) 84 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.449287+0000 mon.a (mon.0) 839 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.450157+0000 mon.a (mon.0) 840 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[19734]: audit 2024-02-18T01:23:18.450565+0000 mon.a (mon.0) 841 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:17.938483+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.018613+0000 mon.a (mon.0) 834 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.018682+0000 mon.a (mon.0) 835 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.018751+0000 mon.a (mon.0) 836 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.018812+0000 mon.a (mon.0) 837 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: cluster 2024-02-18T01:23:18.019156+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.278450+0000 mon.c (mon.2) 84 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.449287+0000 mon.a (mon.0) 839 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.450157+0000 mon.a (mon.0) 840 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:19 smithi105 bash[27669]: audit 2024-02-18T01:23:18.450565+0000 mon.a (mon.0) 841 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:15.704219+0000 osd.3 (osd.3) 5 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:15.805250+0000 osd.3 (osd.3) 6 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:16.413116+0000 mon.b (mon.1) 47 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:16.413467+0000 client.admin (client.25051) 0 : cluster [INF] threexx 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:16.503961+0000 osd.1 (osd.1) 5 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:16.508410+0000 osd.1 (osd.1) 6 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:16.925140+0000 mgr.y (mgr.24475) 244 : cluster [DBG] pgmap v185: 548 pgs: 356 active+clean, 192 unknown; 9.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.118152+0000 mon.a (mon.0) 830 : audit [INF] from='client.25160 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.118369+0000 mon.a (mon.0) 831 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.276839+0000 mon.c (mon.2) 81 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.569100+0000 mgr.y (mgr.24475) 245 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.934702+0000 mon.a (mon.0) 832 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.935561+0000 mon.b (mon.1) 48 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.936498+0000 mon.c (mon.2) 82 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:17.937239+0000 mon.a (mon.0) 833 : cluster [DBG] osdmap e73: 8 total, 8 up, 8 in 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.937398+0000 mon.b (mon.1) 49 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.938073+0000 mon.c (mon.2) 83 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:17.938483+0000 mon.b (mon.1) 50 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.018613+0000 mon.a (mon.0) 834 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi105-61946-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.018682+0000 mon.a (mon.0) 835 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi105-61672-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.018751+0000 mon.a (mon.0) 836 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi105-61665-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.018812+0000 mon.a (mon.0) 837 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi105-61952-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: cluster 2024-02-18T01:23:18.019156+0000 mon.a (mon.0) 838 : cluster [DBG] osdmap e74: 8 total, 8 up, 8 in 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.278450+0000 mon.c (mon.2) 84 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.449287+0000 mon.a (mon.0) 839 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.450157+0000 mon.a (mon.0) 840 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:19.748 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:19 smithi155 bash[20805]: audit 2024-02-18T01:23:18.450565+0000 mon.a (mon.0) 841 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.450157+0000 mon.a [INF] from='client.24797 ' entity='clie delete_pools_parallel: process_1_[62348]: starting. 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: creating pool ceph_test_rados_delete_pools_parallel.smithi105-62322 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: created object 0... 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: created object 25... 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: created object 49... 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: finishing. 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_1_[62348]: shutting down. 2024-02-18T01:23:20.363 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_2_[62349]: starting. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_2_[62349]: deleting pool ceph_test_rados_delete_pools_parallel.smithi105-62322 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_2_[62349]: shutting down. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******************************* 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: starting. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: creating pool ceph_test_rados_delete_pools_parallel.smithi105-62322 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: created object 0... 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: created object 25... 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: created object 49... 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: finishing. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_3_[63330]: shutting down. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******************************* 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: starting. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: listing objects. 2024-02-18T01:23:20.364 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: listed object 0... 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: listed object 25... 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: saw 50 objects 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_5_[63332]: shutting down. 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******************************* 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_4_[63331]: starting. 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_4_[63331]: deleting pool ceph_test_rados_delete_pools_parallel.smithi105-62322 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: process_4_[63331]: shutting down. 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******************************* 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******************************* 2024-02-18T01:23:20.365 INFO:tasks.workunit.client.0.smithi105.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-02-18T01:23:20.463 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: Running main() from gmock_main.cc 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] Global test environment set-up. 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: seed 61672 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (104 ms) 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (10 ms) 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-02-18T01:23:20.464 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (5 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (5 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (3 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (2 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (5 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-02-18T01:23:20.465 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (2 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (4 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (10 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (5 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (3 ms) 2024-02-18T01:23:20.466 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (4 ms) 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (182 ms total) 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (2415 ms) 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (15 ms) 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-02-18T01:23:20.467 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-02-18T01:23:20.468 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-02-18T01:23:20.468 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-02-18T01:23:20.468 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-02-18T01:23:20.468 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (3 ms) 2024-02-18T01:23:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: cluster 2024-02-18T01:23:18.926431+0000 mgr.y (mgr.24475) 246 : cluster [DBG] pgmap v187: 580 pgs: 87 creating+peering, 452 active+clean, 41 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.6 MiB/s rd, 30 MiB/s wr, 225 op/s 2024-02-18T01:23:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.280104+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.322652+0000 mon.a (mon.0) 842 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.330003+0000 mon.a (mon.0) 843 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.330242+0000 mon.a (mon.0) 844 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.344502+0000 mon.c (mon.2) 86 : audit [INF] from='client.? v1:172.21.15.105:0/408252331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.796708+0000 mon.a (mon.0) 845 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.797878+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: cluster 2024-02-18T01:23:19.798358+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.799182+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.799455+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.105:0/4269549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.800414+0000 mon.a (mon.0) 847 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.801835+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:19.801892+0000 mon.a (mon.0) 848 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: cluster 2024-02-18T01:23:19.802184+0000 client.admin (client.25051) 0 : cluster [INF] fourxx 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: cluster 2024-02-18T01:23:18.926431+0000 mgr.y (mgr.24475) 246 : cluster [DBG] pgmap v187: 580 pgs: 87 creating+peering, 452 active+clean, 41 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.6 MiB/s rd, 30 MiB/s wr, 225 op/s 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.280104+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.322652+0000 mon.a (mon.0) 842 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.330003+0000 mon.a (mon.0) 843 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.330242+0000 mon.a (mon.0) 844 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.344502+0000 mon.c (mon.2) 86 : audit [INF] from='client.? v1:172.21.15.105:0/408252331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.796708+0000 mon.a (mon.0) 845 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.797878+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: cluster 2024-02-18T01:23:19.798358+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.799182+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.799455+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.105:0/4269549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.800414+0000 mon.a (mon.0) 847 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T01:23:20.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.801835+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:19.801892+0000 mon.a (mon.0) 848 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: cluster 2024-02-18T01:23:19.802184+0000 client.admin (client.25051) 0 : cluster [INF] fourxx 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:20.249111+0000 mon.a (mon.0) 849 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:20.249809+0000 mon.a (mon.0) 850 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:20.250402+0000 mon.a (mon.0) 851 : audit [INF] from='client.25199 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:20.250913+0000 mon.a (mon.0) 852 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[19734]: audit 2024-02-18T01:23:20.281803+0000 mon.c (mon.2) 89 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:20.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:20.249111+0000 mon.a (mon.0) 849 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:20.249809+0000 mon.a (mon.0) 850 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:20.250402+0000 mon.a (mon.0) 851 : audit [INF] from='client.25199 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:20.250913+0000 mon.a (mon.0) 852 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:20 smithi105 bash[27669]: audit 2024-02-18T01:23:20.281803+0000 mon.c (mon.2) 89 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: cluster 2024-02-18T01:23:18.926431+0000 mgr.y (mgr.24475) 246 : cluster [DBG] pgmap v187: 580 pgs: 87 creating+peering, 452 active+clean, 41 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 9.6 MiB/s rd, 30 MiB/s wr, 225 op/s 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.280104+0000 mon.c (mon.2) 85 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.322652+0000 mon.a (mon.0) 842 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.330003+0000 mon.a (mon.0) 843 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.330242+0000 mon.a (mon.0) 844 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.344502+0000 mon.c (mon.2) 86 : audit [INF] from='client.? v1:172.21.15.105:0/408252331' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.796708+0000 mon.a (mon.0) 845 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.797878+0000 mon.c (mon.2) 87 : audit [INF] from='client.? v1:172.21.15.105:0/4283856707' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: cluster 2024-02-18T01:23:19.798358+0000 mon.a (mon.0) 846 : cluster [DBG] osdmap e75: 8 total, 8 up, 8 in 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.799182+0000 mon.b (mon.1) 51 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.799455+0000 mon.c (mon.2) 88 : audit [INF] from='client.? v1:172.21.15.105:0/4269549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.800414+0000 mon.a (mon.0) 847 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.801835+0000 mon.b (mon.1) 52 : audit [INF] from='client.? v1:172.21.15.105:0/2778648279' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:19.801892+0000 mon.a (mon.0) 848 : audit [INF] from='client.25196 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: cluster 2024-02-18T01:23:19.802184+0000 client.admin (client.25051) 0 : cluster [INF] fourxx 2024-02-18T01:23:20.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:20.249111+0000 mon.a (mon.0) 849 : audit [INF] from='client.24797 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:20.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:20.249809+0000 mon.a (mon.0) 850 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:20.250402+0000 mon.a (mon.0) 851 : audit [INF] from='client.25199 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:20.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:20.250913+0000 mon.a (mon.0) 852 : audit [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-02-18T01:23:20.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:20 smithi155 bash[20805]: audit 2024-02-18T01:23:20.281803+0000 mon.c (mon.2) 89 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: cluster 2024-02-18T01:23:19.951666+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: cluster 2024-02-18T01:23:19.955255+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.344641+0000 mon.a (mon.0) 853 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T01:23:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.351788+0000 mon.a (mon.0) 854 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.351996+0000 mon.a (mon.0) 855 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352122+0000 mon.a (mon.0) 856 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352225+0000 mon.a (mon.0) 857 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352358+0000 mon.a (mon.0) 858 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352705+0000 mon.a (mon.0) 859 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352808+0000 mon.a (mon.0) 860 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.352898+0000 mon.a (mon.0) 861 : audit [INF] from='client.25199 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: cluster 2024-02-18T01:23:20.352985+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.363674+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.364254+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.105:0/2318466880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.364760+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: cluster 2024-02-18T01:23:19.951666+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: cluster 2024-02-18T01:23:19.955255+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.344641+0000 mon.a (mon.0) 853 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.351788+0000 mon.a (mon.0) 854 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.351996+0000 mon.a (mon.0) 855 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352122+0000 mon.a (mon.0) 856 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352225+0000 mon.a (mon.0) 857 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352358+0000 mon.a (mon.0) 858 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352705+0000 mon.a (mon.0) 859 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352808+0000 mon.a (mon.0) 860 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.352898+0000 mon.a (mon.0) 861 : audit [INF] from='client.25199 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: cluster 2024-02-18T01:23:20.352985+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.363674+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.364254+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.105:0/2318466880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.364760+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.365040+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.368523+0000 mon.a (mon.0) 863 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.369412+0000 mon.a (mon.0) 864 : audit [INF] from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.369638+0000 mon.a (mon.0) 865 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.369878+0000 mon.a (mon.0) 866 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.443587+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.444382+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:20.856490+0000 mon.a (mon.0) 867 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: cluster 2024-02-18T01:23:20.927745+0000 mgr.y (mgr.24475) 247 : cluster [DBG] pgmap v190: 652 pgs: 58 creating+peering, 420 active+clean, 174 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 324 op/s 2024-02-18T01:23:21.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[27669]: audit 2024-02-18T01:23:21.283472+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.365040+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.368523+0000 mon.a (mon.0) 863 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.369412+0000 mon.a (mon.0) 864 : audit [INF] from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.369638+0000 mon.a (mon.0) 865 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.369878+0000 mon.a (mon.0) 866 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.443587+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.444382+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:20.856490+0000 mon.a (mon.0) 867 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: cluster 2024-02-18T01:23:20.927745+0000 mgr.y (mgr.24475) 247 : cluster [DBG] pgmap v190: 652 pgs: 58 creating+peering, 420 active+clean, 174 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 324 op/s 2024-02-18T01:23:21.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:21 smithi105 bash[19734]: audit 2024-02-18T01:23:21.283472+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: cluster 2024-02-18T01:23:19.951666+0000 osd.7 (osd.7) 3 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: cluster 2024-02-18T01:23:19.955255+0000 osd.7 (osd.7) 4 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.344641+0000 mon.a (mon.0) 853 : audit [INF] from='client.25051 ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.351788+0000 mon.a (mon.0) 854 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi105-61709-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.351996+0000 mon.a (mon.0) 855 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352122+0000 mon.a (mon.0) 856 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352225+0000 mon.a (mon.0) 857 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi105-61697-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352358+0000 mon.a (mon.0) 858 : audit [INF] from='client.25196 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352705+0000 mon.a (mon.0) 859 : audit [INF] from='client.24797 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi105-61976-1"}]': finished 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352808+0000 mon.a (mon.0) 860 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi105-61657-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.352898+0000 mon.a (mon.0) 861 : audit [INF] from='client.25199 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi105-61649-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: cluster 2024-02-18T01:23:20.352985+0000 mon.a (mon.0) 862 : cluster [DBG] osdmap e76: 8 total, 8 up, 8 in 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.363674+0000 mon.b (mon.1) 53 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.364254+0000 mon.b (mon.1) 54 : audit [INF] from='client.? v1:172.21.15.105:0/2318466880' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.364760+0000 mon.b (mon.1) 55 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.365040+0000 mon.b (mon.1) 56 : audit [INF] from='client.? v1:172.21.15.105:0/2994450003' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.368523+0000 mon.a (mon.0) 863 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.369412+0000 mon.a (mon.0) 864 : audit [INF] from='client.25231 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.369638+0000 mon.a (mon.0) 865 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.369878+0000 mon.a (mon.0) 866 : audit [INF] from='client.25171 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.443587+0000 mon.b (mon.1) 57 : audit [DBG] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.444382+0000 mon.b (mon.1) 58 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:20.856490+0000 mon.a (mon.0) 867 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:21.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: cluster 2024-02-18T01:23:20.927745+0000 mgr.y (mgr.24475) 247 : cluster [DBG] pgmap v190: 652 pgs: 58 creating+peering, 420 active+clean, 174 unknown; 144 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 10 MiB/s rd, 32 MiB/s wr, 324 op/s 2024-02-18T01:23:21.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[20805]: audit 2024-02-18T01:23:21.283472+0000 mon.c (mon.2) 90 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ]nt.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi105-61976-1"}]: dispatch 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: got: 2024-02-18T01:23:18.450565+0000 mon.a [INF] from='client.25051 ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (10915 ms) 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (22239 ms total) 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [----------] Global test environment tear-down 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (22239 ms total) 2024-02-18T01:23:22.349 INFO:tasks.workunit.client.0.smithi105.stdout: api_cmd: [ PASSED ] 4 tests. 2024-02-18T01:23:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: cluster 2024-02-18T01:23:21.112044+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: cluster 2024-02-18T01:23:21.116437+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.802120+0000 mon.a (mon.0) 868 : audit [INF] from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.802331+0000 mon.a (mon.0) 869 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.802501+0000 mon.a (mon.0) 870 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.803496+0000 mon.a (mon.0) 871 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: cluster 2024-02-18T01:23:21.803788+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.804004+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.805538+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.805833+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.806546+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.806934+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.807355+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.807857+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:21.814816+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: cluster 2024-02-18T01:23:21.112044+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: cluster 2024-02-18T01:23:21.116437+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.802120+0000 mon.a (mon.0) 868 : audit [INF] from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.802331+0000 mon.a (mon.0) 869 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.802501+0000 mon.a (mon.0) 870 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.803496+0000 mon.a (mon.0) 871 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: cluster 2024-02-18T01:23:21.803788+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.804004+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.805538+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.805833+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.806546+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.806934+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.807355+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.807857+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:21.814816+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.232036+0000 mon.a (mon.0) 873 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.232419+0000 mon.a (mon.0) 874 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.233182+0000 mon.a (mon.0) 875 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.234510+0000 mon.a (mon.0) 876 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.234868+0000 mon.a (mon.0) 877 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.235195+0000 mon.a (mon.0) 878 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.235541+0000 mon.a (mon.0) 879 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.236074+0000 mon.a (mon.0) 880 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.238577+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.239073+0000 mon.a (mon.0) 881 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.240693+0000 mon.c (mon.2) 94 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.240990+0000 mon.a (mon.0) 882 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[27669]: audit 2024-02-18T01:23:22.241604+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.232036+0000 mon.a (mon.0) 873 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.232419+0000 mon.a (mon.0) 874 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.233182+0000 mon.a (mon.0) 875 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.234510+0000 mon.a (mon.0) 876 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.234868+0000 mon.a (mon.0) 877 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.235195+0000 mon.a (mon.0) 878 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.235541+0000 mon.a (mon.0) 879 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.236074+0000 mon.a (mon.0) 880 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.238577+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.239073+0000 mon.a (mon.0) 881 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.240693+0000 mon.c (mon.2) 94 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.240990+0000 mon.a (mon.0) 882 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:22 smithi105 bash[19734]: audit 2024-02-18T01:23:22.241604+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: cluster 2024-02-18T01:23:21.112044+0000 osd.0 (osd.0) 3 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: cluster 2024-02-18T01:23:21.116437+0000 osd.0 (osd.0) 4 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.802120+0000 mon.a (mon.0) 868 : audit [INF] from='client.25231 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.802331+0000 mon.a (mon.0) 869 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.802501+0000 mon.a (mon.0) 870 : audit [INF] from='client.25171 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi105-61657-3", "field": "max_bytes", "val": "4096"}]': finished 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.803496+0000 mon.a (mon.0) 871 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: cluster 2024-02-18T01:23:21.803788+0000 mon.a (mon.0) 872 : cluster [DBG] osdmap e77: 8 total, 8 up, 8 in 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.804004+0000 mon.b (mon.1) 59 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.805538+0000 mon.c (mon.2) 91 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.805833+0000 mon.b (mon.1) 60 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.806546+0000 mon.c (mon.2) 92 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.806934+0000 mon.b (mon.1) 61 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.807355+0000 mon.b (mon.1) 62 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.807857+0000 mon.b (mon.1) 63 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:21.814816+0000 mon.b (mon.1) 64 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.232036+0000 mon.a (mon.0) 873 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.232419+0000 mon.a (mon.0) 874 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.233182+0000 mon.a (mon.0) 875 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.234510+0000 mon.a (mon.0) 876 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.234868+0000 mon.a (mon.0) 877 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.235195+0000 mon.a (mon.0) 878 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.235541+0000 mon.a (mon.0) 879 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:22.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.236074+0000 mon.a (mon.0) 880 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:22.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.238577+0000 mon.c (mon.2) 93 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.239073+0000 mon.a (mon.0) 881 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.240693+0000 mon.c (mon.2) 94 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.240990+0000 mon.a (mon.0) 882 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:22.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:22 smithi155 bash[20805]: audit 2024-02-18T01:23:22.241604+0000 mon.c (mon.2) 95 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:23.179 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-02-18T01:23:23.179 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-02-18T01:23:23.179 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: seed 61946 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (280 ms) 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (42 ms) 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (18 ms) 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-02-18T01:23:23.180 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (18 ms) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (358 ms total) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (2412 ms) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (7 ms) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (2423 ms total) 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: 2024-02-18T01:23:23.181 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-02-18T01:23:23.182 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (23099 ms total) 2024-02-18T01:23:23.182 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-02-18T01:23:23.195 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: Running main() from gmock_main.cc 2024-02-18T01:23:23.195 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [==========] Running 23 tests from 2 test suites. 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] Global test environment set-up. 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] 13 tests from LibRadosIo 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (84 ms) 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: no timeout :/ 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: no timeout :/ 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: no timeout :/ 2024-02-18T01:23:23.196 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: no timeout :/ 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: no timeout :/ 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (38 ms) 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (6 ms) 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (6 ms) 2024-02-18T01:23:23.197 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (4 ms) 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (4 ms) 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-02-18T01:23:23.198 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (4 ms) 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.RmXattr (12 ms) 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIo.XattrIter (3 ms) 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] 13 tests from LibRadosIo (171 ms total) 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-02-18T01:23:23.199 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2407 ms) 2024-02-18T01:23:23.200 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-02-18T01:23:23.200 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (7 ms) 2024-02-18T01:23:23.200 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5 ms) 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (8 ms) 2024-02-18T01:23:23.201 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (5 ms) 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (3 ms) 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (11 ms) 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (5 ms) 2024-02-18T01:23:23.202 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2460 ms total) 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [----------] Global test environment tear-down 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [==========] 23 tests from 2 test suites ran. (23246 ms total) 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (6 ms) 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-02-18T01:23:23.203 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (7 ms) 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (4 ms) 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (5 ms) 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-02-18T01:23:23.204 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (2 ms) 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (10 ms) 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (3 ms) 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (8 ms) 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-02-18T01:23:23.205 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (5 ms) 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (8 ms) 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (2508 ms total) 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [----------] Global test environment tear-down 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (23242 ms total) 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: Running main() from gmock_main.cc 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] Global test environment set-up. 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-02-18T01:23:23.206 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStat.Stat (343 ms) 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStat.StatNS (15 ms) 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (2 ms) 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] 4 tests from LibRadosStat (362 ms total) 2024-02-18T01:23:23.207 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1397 ms) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (1017 ms) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (3 ms) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (2417 ms total) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [----------] Global test environment tear-down 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (23119 ms total) 2024-02-18T01:23:23.208 INFO:tasks.workunit.client.0.smithi105.stdout: api_stat: [ PASSED ] 8 tests. 2024-02-18T01:23:23.209 INFO:tasks.workunit.client.0.smithi105.stdout: api_io: [ PASSED ] 23 tests. 2024-02-18T01:23:23.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.285167+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:23.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.670819+0000 mon.a (mon.0) 883 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:23.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.679125+0000 mon.a (mon.0) 884 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.679815+0000 mon.a (mon.0) 885 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.679921+0000 mon.a (mon.0) 886 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.680071+0000 mon.a (mon.0) 887 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.680200+0000 mon.a (mon.0) 888 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.680295+0000 mon.a (mon.0) 889 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.680410+0000 mon.a (mon.0) 890 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:23.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.680638+0000 mon.a (mon.0) 891 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:23.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: cluster 2024-02-18T01:23:22.681380+0000 mon.a (mon.0) 892 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T01:23:23.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.681788+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:23.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.682293+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:23.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.682629+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.285167+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.670819+0000 mon.a (mon.0) 883 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.679125+0000 mon.a (mon.0) 884 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.679815+0000 mon.a (mon.0) 885 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.679921+0000 mon.a (mon.0) 886 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:23.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.680071+0000 mon.a (mon.0) 887 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.680200+0000 mon.a (mon.0) 888 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.680295+0000 mon.a (mon.0) 889 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.680410+0000 mon.a (mon.0) 890 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.680638+0000 mon.a (mon.0) 891 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: cluster 2024-02-18T01:23:22.681380+0000 mon.a (mon.0) 892 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T01:23:23.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.681788+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:23.998 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.682293+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:23.998 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.682629+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:23.998 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.683048+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:23.998 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.684116+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T01:23:23.998 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.686470+0000 mon.a (mon.0) 893 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:23.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.686760+0000 mon.a (mon.0) 894 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:23.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.686947+0000 mon.a (mon.0) 895 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:23.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.687157+0000 mon.a (mon.0) 896 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:23.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.689698+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:23.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.689737+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.689968+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2024487925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.692106+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.105:0/3724249021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.694078+0000 mon.a (mon.0) 897 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.714020+0000 mon.a (mon.0) 898 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:24.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.714381+0000 mon.a (mon.0) 899 : audit [INF] from='client.25220 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.715403+0000 mon.a (mon.0) 900 : audit [INF] from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:22.735701+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.683048+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.684116+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.686470+0000 mon.a (mon.0) 893 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:24.001 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.686760+0000 mon.a (mon.0) 894 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.686947+0000 mon.a (mon.0) 895 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.687157+0000 mon.a (mon.0) 896 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.689698+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.689737+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.689968+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2024487925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.692106+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.105:0/3724249021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.694078+0000 mon.a (mon.0) 897 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.714020+0000 mon.a (mon.0) 898 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.714381+0000 mon.a (mon.0) 899 : audit [INF] from='client.25220 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.002 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.715403+0000 mon.a (mon.0) 900 : audit [INF] from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:22.735701+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: cluster 2024-02-18T01:23:22.929042+0000 mgr.y (mgr.24475) 248 : cluster [DBG] pgmap v193: 596 pgs: 32 creating+peering, 492 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: cluster 2024-02-18T01:23:23.159198+0000 mon.a (mon.0) 901 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: cluster 2024-02-18T01:23:23.163634+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.173906+0000 mon.a (mon.0) 903 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174188+0000 mon.a (mon.0) 904 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174297+0000 mon.a (mon.0) 905 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]': finished 2024-02-18T01:23:24.003 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174400+0000 mon.a (mon.0) 906 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174490+0000 mon.a (mon.0) 907 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174616+0000 mon.a (mon.0) 908 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174730+0000 mon.a (mon.0) 909 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.174982+0000 mon.a (mon.0) 910 : audit [INF] from='client.25220 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.175432+0000 mon.c (mon.2) 101 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.176944+0000 mon.a (mon.0) 911 : audit [INF] from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: cluster 2024-02-18T01:23:23.177217+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T01:23:24.004 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.177528+0000 mon.a (mon.0) 913 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: cluster 2024-02-18T01:23:22.929042+0000 mgr.y (mgr.24475) 248 : cluster [DBG] pgmap v193: 596 pgs: 32 creating+peering, 492 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: cluster 2024-02-18T01:23:23.159198+0000 mon.a (mon.0) 901 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: cluster 2024-02-18T01:23:23.163634+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.173906+0000 mon.a (mon.0) 903 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174188+0000 mon.a (mon.0) 904 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174297+0000 mon.a (mon.0) 905 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]': finished 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174400+0000 mon.a (mon.0) 906 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:24.005 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174490+0000 mon.a (mon.0) 907 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174616+0000 mon.a (mon.0) 908 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174730+0000 mon.a (mon.0) 909 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.174982+0000 mon.a (mon.0) 910 : audit [INF] from='client.25220 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.175432+0000 mon.c (mon.2) 101 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.176944+0000 mon.a (mon.0) 911 : audit [INF] from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: cluster 2024-02-18T01:23:23.177217+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.177528+0000 mon.a (mon.0) 913 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.006 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.195233+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.238292+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.286530+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.619130+0000 mon.a (mon.0) 914 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.619750+0000 mon.a (mon.0) 915 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.620647+0000 mon.a (mon.0) 916 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[27669]: audit 2024-02-18T01:23:23.626535+0000 mon.a (mon.0) 917 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:24.007 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.195233+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.238292+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:24.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.286530+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:24.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.619130+0000 mon.a (mon.0) 914 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.619750+0000 mon.a (mon.0) 915 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.008 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.620647+0000 mon.a (mon.0) 916 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.009 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:23 smithi105 bash[19734]: audit 2024-02-18T01:23:23.626535+0000 mon.a (mon.0) 917 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.285167+0000 mon.c (mon.2) 96 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.670819+0000 mon.a (mon.0) 883 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.679125+0000 mon.a (mon.0) 884 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi105-61697-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.679815+0000 mon.a (mon.0) 885 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.679921+0000 mon.a (mon.0) 886 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.680071+0000 mon.a (mon.0) 887 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-4", "overlaypool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.680200+0000 mon.a (mon.0) 888 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.680295+0000 mon.a (mon.0) 889 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.680410+0000 mon.a (mon.0) 890 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:24.010 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.680638+0000 mon.a (mon.0) 891 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi105-61707-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.011 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: cluster 2024-02-18T01:23:22.681380+0000 mon.a (mon.0) 892 : cluster [DBG] osdmap e78: 8 total, 8 up, 8 in 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.681788+0000 mon.b (mon.1) 65 : audit [INF] from='client.? v1:172.21.15.105:0/4039936605' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.682293+0000 mon.b (mon.1) 66 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.682629+0000 mon.b (mon.1) 67 : audit [INF] from='client.? v1:172.21.15.105:0/2709957568' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.683048+0000 mon.b (mon.1) 68 : audit [INF] from='client.? v1:172.21.15.105:0/2940848146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.684116+0000 mon.b (mon.1) 69 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.686470+0000 mon.a (mon.0) 893 : audit [INF] from='client.25045 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.686760+0000 mon.a (mon.0) 894 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.686947+0000 mon.a (mon.0) 895 : audit [INF] from='client.25183 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.687157+0000 mon.a (mon.0) 896 : audit [INF] from='client.25042 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.689698+0000 mon.c (mon.2) 97 : audit [INF] from='client.? v1:172.21.15.105:0/4271791913' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:24.012 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.689737+0000 mon.b (mon.1) 70 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.689968+0000 mon.c (mon.2) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2024487925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.692106+0000 mon.c (mon.2) 99 : audit [INF] from='client.? v1:172.21.15.105:0/3724249021' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.694078+0000 mon.a (mon.0) 897 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.714020+0000 mon.a (mon.0) 898 : audit [INF] from='client.25163 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.714381+0000 mon.a (mon.0) 899 : audit [INF] from='client.25220 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.715403+0000 mon.a (mon.0) 900 : audit [INF] from='client.25229 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:22.735701+0000 mon.c (mon.2) 100 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: cluster 2024-02-18T01:23:22.929042+0000 mgr.y (mgr.24475) 248 : cluster [DBG] pgmap v193: 596 pgs: 32 creating+peering, 492 active+clean, 72 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: cluster 2024-02-18T01:23:23.159198+0000 mon.a (mon.0) 901 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: cluster 2024-02-18T01:23:23.163634+0000 mon.a (mon.0) 902 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.173906+0000 mon.a (mon.0) 903 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174188+0000 mon.a (mon.0) 904 : audit [INF] from='client.25045 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi105-61665-15"}]': finished 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174297+0000 mon.a (mon.0) 905 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-4-cache", "mode": "writeback"}]': finished 2024-02-18T01:23:24.013 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174400+0000 mon.a (mon.0) 906 : audit [INF] from='client.25183 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi105-61952-6"}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174490+0000 mon.a (mon.0) 907 : audit [INF] from='client.25042 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi105-61672-22"}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174616+0000 mon.a (mon.0) 908 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174730+0000 mon.a (mon.0) 909 : audit [INF] from='client.25163 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi105-61946-7"}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.174982+0000 mon.a (mon.0) 910 : audit [INF] from='client.25220 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.175432+0000 mon.c (mon.2) 101 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.176944+0000 mon.a (mon.0) 911 : audit [INF] from='client.25229 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi105-61649-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: cluster 2024-02-18T01:23:23.177217+0000 mon.a (mon.0) 912 : cluster [DBG] osdmap e79: 8 total, 8 up, 8 in 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.177528+0000 mon.a (mon.0) 913 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.195233+0000 mon.b (mon.1) 71 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.238292+0000 mon.b (mon.1) 72 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.286530+0000 mon.c (mon.2) 102 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.619130+0000 mon.a (mon.0) 914 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.619750+0000 mon.a (mon.0) 915 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.620647+0000 mon.a (mon.0) 916 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:23:24.014 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:23 smithi155 bash[20805]: audit 2024-02-18T01:23:23.626535+0000 mon.a (mon.0) 917 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]: dispatch 2024-02-18T01:23:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: cluster 2024-02-18T01:23:24.126675+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: cluster 2024-02-18T01:23:24.129837+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.187092+0000 mon.a (mon.0) 918 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]': finished 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.189409+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.287881+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.614174+0000 mon.a (mon.0) 919 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.614940+0000 mon.a (mon.0) 920 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.615456+0000 mon.a (mon.0) 921 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]': finished 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: cluster 2024-02-18T01:23:24.615693+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.617166+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.617730+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.618422+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.619622+0000 mon.a (mon.0) 923 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.623077+0000 mon.a (mon.0) 924 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.623406+0000 mon.a (mon.0) 925 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:25 smithi155 bash[20805]: audit 2024-02-18T01:23:24.623861+0000 mon.a (mon.0) 926 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: cluster 2024-02-18T01:23:24.126675+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: cluster 2024-02-18T01:23:24.129837+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.187092+0000 mon.a (mon.0) 918 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]': finished 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.189409+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.287881+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.614174+0000 mon.a (mon.0) 919 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.614940+0000 mon.a (mon.0) 920 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:23:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.615456+0000 mon.a (mon.0) 921 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]': finished 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: cluster 2024-02-18T01:23:24.615693+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.617166+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.617730+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.618422+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.619622+0000 mon.a (mon.0) 923 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.623077+0000 mon.a (mon.0) 924 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.623406+0000 mon.a (mon.0) 925 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[19734]: audit 2024-02-18T01:23:24.623861+0000 mon.a (mon.0) 926 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: cluster 2024-02-18T01:23:24.126675+0000 osd.0 (osd.0) 5 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: cluster 2024-02-18T01:23:24.129837+0000 osd.0 (osd.0) 6 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.187092+0000 mon.a (mon.0) 918 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pg_num": 4}]': finished 2024-02-18T01:23:26.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.189409+0000 mon.c (mon.2) 103 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.287881+0000 mon.c (mon.2) 104 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.614174+0000 mon.a (mon.0) 919 : audit [INF] from='client.? v1:172.21.15.105:0/1090916713' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.614940+0000 mon.a (mon.0) 920 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.615456+0000 mon.a (mon.0) 921 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-4"}]': finished 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: cluster 2024-02-18T01:23:24.615693+0000 mon.a (mon.0) 922 : cluster [DBG] osdmap e80: 8 total, 8 up, 8 in 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.617166+0000 mon.b (mon.1) 73 : audit [INF] from='client.? v1:172.21.15.105:0/1031521440' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.617730+0000 mon.b (mon.1) 74 : audit [INF] from='client.? v1:172.21.15.105:0/371354039' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.618422+0000 mon.c (mon.2) 105 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.619622+0000 mon.a (mon.0) 923 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.623077+0000 mon.a (mon.0) 924 : audit [INF] from='client.25069 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.623406+0000 mon.a (mon.0) 925 : audit [INF] from='client.25213 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:23:26.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:25 smithi105 bash[27669]: audit 2024-02-18T01:23:24.623861+0000 mon.a (mon.0) 926 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:26.519 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: seed 61709 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (284 ms) 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (13 ms) 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1040 ms) 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-02-18T01:23:26.520 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1006 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (5 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (8 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (97 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (6 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2459 ms total) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1100 ms) 2024-02-18T01:23:26.521 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (5 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1005 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1009 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (5 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (5 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (4 ms) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (3137 ms total) 2024-02-18T01:23:26.522 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: 2024-02-18T01:23:26.523 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-02-18T01:23:26.523 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (26539 ms total) 2024-02-18T01:23:26.523 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-02-18T01:23:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:26] "GET /metrics HTTP/1.1" 200 208244 "" "Prometheus/2.33.4" 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:24.930254+0000 mgr.y (mgr.24475) 249 : cluster [DBG] pgmap v196: 528 pgs: 32 creating+peering, 452 active+clean, 44 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.081696+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.083757+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.288886+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.492388+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.502858+0000 mon.a (mon.0) 928 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.503326+0000 mon.a (mon.0) 929 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.504097+0000 mon.a (mon.0) 930 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.504266+0000 mon.a (mon.0) 931 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.504426+0000 mon.a (mon.0) 932 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.507301+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.510261+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.510784+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.105:0/1170874857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.511308+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.105:0/1490003247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.903148+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:24.930254+0000 mgr.y (mgr.24475) 249 : cluster [DBG] pgmap v196: 528 pgs: 32 creating+peering, 452 active+clean, 44 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.081696+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.083757+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:27.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.288886+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.492388+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.502858+0000 mon.a (mon.0) 928 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.503326+0000 mon.a (mon.0) 929 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.504097+0000 mon.a (mon.0) 930 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.504266+0000 mon.a (mon.0) 931 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.504426+0000 mon.a (mon.0) 932 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.507301+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.510261+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.510784+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.105:0/1170874857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.511308+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.105:0/1490003247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.903148+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.906541+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:25.982511+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.987182+0000 mon.a (mon.0) 934 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.987657+0000 mon.a (mon.0) 935 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.988008+0000 mon.a (mon.0) 936 : audit [INF] from='client.25253 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:25.988330+0000 mon.a (mon.0) 937 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.289849+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.505857+0000 mon.a (mon.0) 938 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.505994+0000 mon.a (mon.0) 939 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.506077+0000 mon.a (mon.0) 940 : audit [INF] from='client.25253 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.506202+0000 mon.a (mon.0) 941 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:26.506336+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.510355+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.513612+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.105:0/1601183451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.514294+0000 mon.a (mon.0) 943 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.517940+0000 mon.a (mon.0) 944 : audit [INF] from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.520406+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.523533+0000 mon.a (mon.0) 945 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.525676+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3404408422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: audit 2024-02-18T01:23:26.527922+0000 mon.a (mon.0) 946 : audit [INF] from='client.25262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[27669]: cluster 2024-02-18T01:23:26.931025+0000 mgr.y (mgr.24475) 250 : cluster [DBG] pgmap v199: 592 pgs: 32 creating+peering, 420 active+clean, 140 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.906541+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:25.982511+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.987182+0000 mon.a (mon.0) 934 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.987657+0000 mon.a (mon.0) 935 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.495 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.988008+0000 mon.a (mon.0) 936 : audit [INF] from='client.25253 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:25.988330+0000 mon.a (mon.0) 937 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.289849+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.505857+0000 mon.a (mon.0) 938 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.505994+0000 mon.a (mon.0) 939 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.506077+0000 mon.a (mon.0) 940 : audit [INF] from='client.25253 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.506202+0000 mon.a (mon.0) 941 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:26.506336+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.510355+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.513612+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.105:0/1601183451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.514294+0000 mon.a (mon.0) 943 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.517940+0000 mon.a (mon.0) 944 : audit [INF] from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.520406+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.496 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.523533+0000 mon.a (mon.0) 945 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.525676+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3404408422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: audit 2024-02-18T01:23:26.527922+0000 mon.a (mon.0) 946 : audit [INF] from='client.25262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.497 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:26 smithi105 bash[19734]: cluster 2024-02-18T01:23:26.931025+0000 mgr.y (mgr.24475) 250 : cluster [DBG] pgmap v199: 592 pgs: 32 creating+peering, 420 active+clean, 140 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:27.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:24.930254+0000 mgr.y (mgr.24475) 249 : cluster [DBG] pgmap v196: 528 pgs: 32 creating+peering, 452 active+clean, 44 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-02-18T01:23:27.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.081696+0000 osd.0 (osd.0) 7 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.083757+0000 osd.0 (osd.0) 8 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.288886+0000 mon.c (mon.2) 106 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.492388+0000 mon.a (mon.0) 927 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.502858+0000 mon.a (mon.0) 928 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi105-61957-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.503326+0000 mon.a (mon.0) 929 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.504097+0000 mon.a (mon.0) 930 : audit [INF] from='client.25069 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-4", "tierpool": "test-rados-api-smithi105-61833-4-cache"}]': finished 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.504266+0000 mon.a (mon.0) 931 : audit [INF] from='client.25213 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi105-61707-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.504426+0000 mon.a (mon.0) 932 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.507301+0000 mon.c (mon.2) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.510261+0000 mon.c (mon.2) 108 : audit [INF] from='client.? v1:172.21.15.105:0/952076599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.510784+0000 mon.c (mon.2) 109 : audit [INF] from='client.? v1:172.21.15.105:0/1170874857' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.511308+0000 mon.c (mon.2) 110 : audit [INF] from='client.? v1:172.21.15.105:0/1490003247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.903148+0000 osd.7 (osd.7) 5 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:27.498 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.906541+0000 osd.7 (osd.7) 6 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:25.982511+0000 mon.a (mon.0) 933 : cluster [DBG] osdmap e81: 8 total, 8 up, 8 in 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.987182+0000 mon.a (mon.0) 934 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.987657+0000 mon.a (mon.0) 935 : audit [INF] from='client.25169 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.988008+0000 mon.a (mon.0) 936 : audit [INF] from='client.25253 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:25.988330+0000 mon.a (mon.0) 937 : audit [INF] from='client.25256 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.289849+0000 mon.c (mon.2) 111 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.505857+0000 mon.a (mon.0) 938 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.505994+0000 mon.a (mon.0) 939 : audit [INF] from='client.25169 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi105-61709-10"}]': finished 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.506077+0000 mon.a (mon.0) 940 : audit [INF] from='client.25253 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi105-61891-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.506202+0000 mon.a (mon.0) 941 : audit [INF] from='client.25256 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi105-61649-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: cluster 2024-02-18T01:23:26.506336+0000 mon.a (mon.0) 942 : cluster [DBG] osdmap e82: 8 total, 8 up, 8 in 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.510355+0000 mon.c (mon.2) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.513612+0000 mon.c (mon.2) 113 : audit [INF] from='client.? v1:172.21.15.105:0/1601183451' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.499 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.514294+0000 mon.a (mon.0) 943 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.517940+0000 mon.a (mon.0) 944 : audit [INF] from='client.25283 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:26 smithi155 bash[20805]: audit 2024-02-18T01:23:26.520406+0000 mon.b (mon.1) 75 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:27 smithi155 bash[20805]: audit 2024-02-18T01:23:26.523533+0000 mon.a (mon.0) 945 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:27 smithi155 bash[20805]: audit 2024-02-18T01:23:26.525676+0000 mon.c (mon.2) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3404408422' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:27 smithi155 bash[20805]: audit 2024-02-18T01:23:26.527922+0000 mon.a (mon.0) 946 : audit [INF] from='client.25262 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:27.500 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:27 smithi155 bash[20805]: cluster 2024-02-18T01:23:26.931025+0000 mgr.y (mgr.24475) 250 : cluster [DBG] pgmap v199: 592 pgs: 32 creating+peering, 420 active+clean, 140 unknown; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:27.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: Running main() from gmock_main.cc 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] Global test environment set-up. 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (208 ms) 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.LockShared (101 ms) 2024-02-18T01:23:28.238 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1016 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1015 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (6 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.Unlock (5 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (6 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (4 ms) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] 8 tests from LibRadosLock (2361 ms total) 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-02-18T01:23:28.239 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (550 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (3 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1006 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1024 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (5 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (6 ms) 2024-02-18T01:23:28.240 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (4 ms) 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (2602 ms total) 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [----------] Global test environment tear-down 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (28269 ms total) 2024-02-18T01:23:28.241 INFO:tasks.workunit.client.0.smithi105.stdout: api_lock: [ PASSED ] 16 tests. 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.290961+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: cluster 2024-02-18T01:23:27.501976+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.514772+0000 mon.a (mon.0) 948 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]': finished 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.515210+0000 mon.a (mon.0) 949 : audit [INF] from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.515351+0000 mon.a (mon.0) 950 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.515453+0000 mon.a (mon.0) 951 : audit [INF] from='client.25262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: cluster 2024-02-18T01:23:27.515640+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T01:23:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.518416+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.523628+0000 mon.a (mon.0) 953 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.530234+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[19734]: audit 2024-02-18T01:23:27.532730+0000 mon.a (mon.0) 954 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.290961+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: cluster 2024-02-18T01:23:27.501976+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.514772+0000 mon.a (mon.0) 948 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]': finished 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.515210+0000 mon.a (mon.0) 949 : audit [INF] from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.515351+0000 mon.a (mon.0) 950 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.515453+0000 mon.a (mon.0) 951 : audit [INF] from='client.25262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: cluster 2024-02-18T01:23:27.515640+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.518416+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.523628+0000 mon.a (mon.0) 953 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.530234+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:28.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:28 smithi105 bash[27669]: audit 2024-02-18T01:23:27.532730+0000 mon.a (mon.0) 954 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.290961+0000 mon.c (mon.2) 115 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: cluster 2024-02-18T01:23:27.501976+0000 mon.a (mon.0) 947 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.514772+0000 mon.a (mon.0) 948 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "mode": "writeback"}]': finished 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.515210+0000 mon.a (mon.0) 949 : audit [INF] from='client.25283 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.515351+0000 mon.a (mon.0) 950 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.515453+0000 mon.a (mon.0) 951 : audit [INF] from='client.25262 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: cluster 2024-02-18T01:23:27.515640+0000 mon.a (mon.0) 952 : cluster [DBG] osdmap e83: 8 total, 8 up, 8 in 2024-02-18T01:23:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.518416+0000 mon.b (mon.1) 76 : audit [INF] from='client.? v1:172.21.15.105:0/615769505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.523628+0000 mon.a (mon.0) 953 : audit [INF] from='client.25210 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]: dispatch 2024-02-18T01:23:28.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.530234+0000 mon.c (mon.2) 116 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:28.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:28 smithi155 bash[20805]: audit 2024-02-18T01:23:27.532730+0000 mon.a (mon.0) 954 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.350 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-02-18T01:23:29.350 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-02-18T01:23:29.350 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify_test_cb 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (325 ms) 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93986814313632 err -107 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (21 ms) 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93986814329616 err -107 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-02-18T01:23:29.351 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (9 ms) 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743425 cookie 93986814335616 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (6 ms) 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743425 cookie 93986814340112 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (8 ms) 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743424 cookie 93986814386880 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743425 cookie 93986814367424 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743425 cookie 93986814404304 2024-02-18T01:23:29.352 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (9 ms) 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 296352743424 cookie 93986814403376 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 300647710721 cookie 93986814403376 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3006 ms) 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93986814403376 err -107 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_cb from 24785 notify_id 322122547204 cookie 93986814403376 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5011 ms) 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify2_test_errcb cookie 93986814441088 err -107 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1007 ms) 2024-02-18T01:23:29.353 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (9408 ms total) 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: watch_notify_test_cb 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1076 ms) 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1076 ms total) 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (29267 ms total) 2024-02-18T01:23:29.354 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:27.577917+0000 mgr.y (mgr.24475) 251 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: cluster 2024-02-18T01:23:28.165918+0000 mon.a (mon.0) 955 : cluster [WRN] pool 'PoolQuotaPP_smithi105-61657-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: cluster 2024-02-18T01:23:28.167815+0000 mon.a (mon.0) 956 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.176000+0000 mon.a (mon.0) 957 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.176085+0000 mon.a (mon.0) 958 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: cluster 2024-02-18T01:23:28.176254+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.178501+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.105:0/4149915667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.180259+0000 mon.a (mon.0) 960 : audit [INF] from='client.26278 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.215637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.217790+0000 mon.a (mon.0) 961 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[27669]: audit 2024-02-18T01:23:28.293427+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:27.577917+0000 mgr.y (mgr.24475) 251 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: cluster 2024-02-18T01:23:28.165918+0000 mon.a (mon.0) 955 : cluster [WRN] pool 'PoolQuotaPP_smithi105-61657-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: cluster 2024-02-18T01:23:28.167815+0000 mon.a (mon.0) 956 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.176000+0000 mon.a (mon.0) 957 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.176085+0000 mon.a (mon.0) 958 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: cluster 2024-02-18T01:23:28.176254+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.178501+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.105:0/4149915667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.180259+0000 mon.a (mon.0) 960 : audit [INF] from='client.26278 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.215637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.217790+0000 mon.a (mon.0) 961 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:29 smithi105 bash[19734]: audit 2024-02-18T01:23:28.293427+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:27.577917+0000 mgr.y (mgr.24475) 251 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: cluster 2024-02-18T01:23:28.165918+0000 mon.a (mon.0) 955 : cluster [WRN] pool 'PoolQuotaPP_smithi105-61657-3' is full (reached quota's max_bytes: 4 KiB) 2024-02-18T01:23:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: cluster 2024-02-18T01:23:28.167815+0000 mon.a (mon.0) 956 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:23:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.176000+0000 mon.a (mon.0) 957 : audit [INF] from='client.25210 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi105-61697-10"}]': finished 2024-02-18T01:23:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.176085+0000 mon.a (mon.0) 958 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: cluster 2024-02-18T01:23:28.176254+0000 mon.a (mon.0) 959 : cluster [DBG] osdmap e84: 8 total, 8 up, 8 in 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.178501+0000 mon.b (mon.1) 77 : audit [INF] from='client.? v1:172.21.15.105:0/4149915667' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.180259+0000 mon.a (mon.0) 960 : audit [INF] from='client.26278 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.215637+0000 mon.c (mon.2) 117 : audit [INF] from='client.? v1:172.21.15.105:0/3509180531' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.217790+0000 mon.a (mon.0) 961 : audit [INF] from='client.25232 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]: dispatch 2024-02-18T01:23:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:29 smithi155 bash[20805]: audit 2024-02-18T01:23:28.293427+0000 mon.c (mon.2) 118 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:30.422 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-02-18T01:23:30.422 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [----------] Global test environment set-up. 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5039 ms) 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (37 ms) 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20046 ms) 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: attempt 0 of 20 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: failed to deregister: 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: { 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "epoch": 18, 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "modified": "2024-02-18T01:23:26.930532+0000", 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "services": { 2024-02-18T01:23:30.423 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "laundry": { 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "daemons": { 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "summary": "", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "close-test-pid62154": { 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_epoch": 18, 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_stamp": "2024-02-18T01:23:25.339090+0000", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "gid": 25268, 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "addr": "172.21.15.105:0/3526067120", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "metadata": { 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro": "ubuntu", 2024-02-18T01:23:30.424 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_description": "Ubuntu 18.04.6 LTS", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_version": "18.04", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "foo": "bar", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "hostname": "smithi105", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_total_kb": "32823852", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "os": "Linux", 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "this": "that" 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "task_status": {} 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "pid62118": { 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_epoch": 17, 2024-02-18T01:23:30.425 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_stamp": "2024-02-18T01:23:11.783022+0000", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "gid": 25063, 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "addr": "172.21.15.105:0/3940888495", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "metadata": { 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro": "ubuntu", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_description": "Ubuntu 18.04.6 LTS", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_version": "18.04", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "foo": "bar", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "hostname": "smithi105", 2024-02-18T01:23:30.426 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_total_kb": "32823852", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "os": "Linux", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "this": "that" 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "task_status": {} 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "rgw": { 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "daemons": { 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "summary": "", 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "24398": { 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_epoch": 13, 2024-02-18T01:23:30.427 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_stamp": "2024-02-18T01:17:17.946941+0000", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "gid": 24398, 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "addr": "172.21.15.105:0/2475373584", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "metadata": { 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "container_hostname": "smithi105", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc0191869e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro": "centos", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_version": "8", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-18T01:23:30.428 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "hostname": "smithi105", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "id": "foo.a", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_total_kb": "32823852", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "num_handles": "1", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "os": "Linux", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "pid": "7", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "realm_id": "", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "realm_name": "", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zone_id": "359ef169-be4f-46cd-ad28-f5ca4f52f983", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zone_name": "default", 2024-02-18T01:23:30.429 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zonegroup_id": "fc6cf9f5-802c-4a69-9859-d03009cee767", 2024-02-18T01:23:30.703 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: api_watch_notify_pp: Running main() from gmock_main.cc 2024-02-18T01:23:30.703 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-02-18T01:23:30.703 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1119 ms) 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (19 ms) 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1138 ms total) 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify 2024-02-18T01:23:30.704 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (45 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (4855 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (18 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285571 notifier_gid 25211 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (13 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285572 notifier_gid 25211 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (9 ms) 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-02-18T01:23:30.705 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285573 notifier_gid 25211 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (21 ms) 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285574 notifier_gid 25211 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (11 ms) 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285575 notifier_gid 25211 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (13 ms) 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285576 notifier_gid 25211 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (10 ms) 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: trying... 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337856 notify_id 356482285577 notifier_gid 25211 2024-02-18T01:23:30.706 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: timed out 2024-02-18T01:23:30.707 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: flushing 2024-02-18T01:23:30.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:30 smithi105 bash[19734]: cluster 2024-02-18T01:23:28.931667+0000 mgr.y (mgr.24475) 252 : cluster [DBG] pgmap v202: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:30 smithi105 bash[19734]: audit 2024-02-18T01:23:29.301689+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:30 smithi105 bash[27669]: cluster 2024-02-18T01:23:28.931667+0000 mgr.y (mgr.24475) 252 : cluster [DBG] pgmap v202: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:30 smithi105 bash[27669]: audit 2024-02-18T01:23:29.301689+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:30 smithi155 bash[20805]: cluster 2024-02-18T01:23:28.931667+0000 mgr.y (mgr.24475) 252 : cluster [DBG] pgmap v202: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:30 smithi155 bash[20805]: audit 2024-02-18T01:23:29.301689+0000 mon.c (mon.2) 119 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:29.320770+0000 mon.a (mon.0) 962 : audit [INF] from='client.26278 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:29.320914+0000 mon.a (mon.0) 963 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:29.329414+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:29.333778+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.105:0/4157713383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.302893+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.382049+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? v1:172.21.15.105:0/64609053' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.382284+0000 mgr.y (mgr.24475) 253 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: cluster 2024-02-18T01:23:30.385761+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.390731+0000 mon.a (mon.0) 965 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.395367+0000 mon.a (mon.0) 966 : audit [INF] from='client.26881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: cluster 2024-02-18T01:23:30.404380+0000 mon.a (mon.0) 967 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.415651+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2149726628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.416271+0000 mon.a (mon.0) 968 : audit [INF] from='client.26882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.420569+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.105:0/1393706341' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:30.422220+0000 mon.a (mon.0) 969 : audit [INF] from='client.26920 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: cluster 2024-02-18T01:23:30.932370+0000 mgr.y (mgr.24475) 254 : cluster [DBG] pgmap v205: 552 pgs: 160 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:31.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[19734]: audit 2024-02-18T01:23:31.303994+0000 mon.c (mon.2) 123 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:29.320770+0000 mon.a (mon.0) 962 : audit [INF] from='client.26278 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:29.320914+0000 mon.a (mon.0) 963 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:29.329414+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:29.333778+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.105:0/4157713383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.302893+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.382049+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? v1:172.21.15.105:0/64609053' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.382284+0000 mgr.y (mgr.24475) 253 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: cluster 2024-02-18T01:23:30.385761+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T01:23:31.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.390731+0000 mon.a (mon.0) 965 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.395367+0000 mon.a (mon.0) 966 : audit [INF] from='client.26881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: cluster 2024-02-18T01:23:30.404380+0000 mon.a (mon.0) 967 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.415651+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2149726628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.416271+0000 mon.a (mon.0) 968 : audit [INF] from='client.26882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.420569+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.105:0/1393706341' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:30.422220+0000 mon.a (mon.0) 969 : audit [INF] from='client.26920 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: cluster 2024-02-18T01:23:30.932370+0000 mgr.y (mgr.24475) 254 : cluster [DBG] pgmap v205: 552 pgs: 160 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:31.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:31 smithi105 bash[27669]: audit 2024-02-18T01:23:31.303994+0000 mon.c (mon.2) 123 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:29.320770+0000 mon.a (mon.0) 962 : audit [INF] from='client.26278 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi105-61649-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:31.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:29.320914+0000 mon.a (mon.0) 963 : audit [INF] from='client.25232 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi105-61957-12"}]': finished 2024-02-18T01:23:31.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:29.329414+0000 mon.b (mon.1) 78 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:29.333778+0000 mon.b (mon.1) 79 : audit [INF] from='client.? v1:172.21.15.105:0/4157713383' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.302893+0000 mon.c (mon.2) 120 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.382049+0000 mon.c (mon.2) 121 : audit [DBG] from='client.? v1:172.21.15.105:0/64609053' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.382284+0000 mgr.y (mgr.24475) 253 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: cluster 2024-02-18T01:23:30.385761+0000 mon.a (mon.0) 964 : cluster [DBG] osdmap e85: 8 total, 8 up, 8 in 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.390731+0000 mon.a (mon.0) 965 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.395367+0000 mon.a (mon.0) 966 : audit [INF] from='client.26881 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: cluster 2024-02-18T01:23:30.404380+0000 mon.a (mon.0) 967 : cluster [DBG] osdmap e86: 8 total, 8 up, 8 in 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.415651+0000 mon.c (mon.2) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2149726628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.416271+0000 mon.a (mon.0) 968 : audit [INF] from='client.26882 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.420569+0000 mon.b (mon.1) 80 : audit [INF] from='client.? v1:172.21.15.105:0/1393706341' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:30.422220+0000 mon.a (mon.0) 969 : audit [INF] from='client.26920 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: cluster 2024-02-18T01:23:30.932370+0000 mgr.y (mgr.24475) 254 : cluster [DBG] pgmap v205: 552 pgs: 160 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:23:31.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[20805]: audit 2024-02-18T01:23:31.303994+0000 mon.c (mon.2) 123 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: ap_service: 17 stray daemon(s) not managed by cephadm 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: services: 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 11m) 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: mgr: y(active, since 5m), standbys: x 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: osd: 8 osds: 8 up (since 6m), 8 in (since 6m) 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-02-18T01:23:31.800 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: data: 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pools: 30 pools, 900 pgs 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: objects: 224 objects, 5.7 KiB 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: pgs: 68.667% pgs unknown 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 16.667% pgs not active 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 618 unknown 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 150 creating+peering 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 132 active+clean 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: io: 2024-02-18T01:23:31.801 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: client: 786 B/s rd, 0 op/s rd, 0 op/s wr 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ OK ] LibRadosService.StatusFormat (6446 ms) 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ RUN ] LibRadosService.Status 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ OK ] LibRadosService.Status (20096 ms) 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [----------] 4 tests from LibRadosService (31643 ms total) 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [----------] Global test environment tear-down 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [==========] 4 tests from 1 test suite ran. (31643 ms total) 2024-02-18T01:23:31.802 INFO:tasks.workunit.client.0.smithi105.stdout: api_service: [ PASSED ] 4 tests. 2024-02-18T01:23:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[27669]: audit 2024-02-18T01:23:31.414265+0000 mon.a (mon.0) 970 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[27669]: audit 2024-02-18T01:23:31.414400+0000 mon.a (mon.0) 971 : audit [INF] from='client.26881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[27669]: audit 2024-02-18T01:23:31.414595+0000 mon.a (mon.0) 972 : audit [INF] from='client.26882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[27669]: audit 2024-02-18T01:23:31.414717+0000 mon.a (mon.0) 973 : audit [INF] from='client.26920 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[27669]: cluster 2024-02-18T01:23:31.414813+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T01:23:32.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[19734]: audit 2024-02-18T01:23:31.414265+0000 mon.a (mon.0) 970 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[19734]: audit 2024-02-18T01:23:31.414400+0000 mon.a (mon.0) 971 : audit [INF] from='client.26881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[19734]: audit 2024-02-18T01:23:31.414595+0000 mon.a (mon.0) 972 : audit [INF] from='client.26882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[19734]: audit 2024-02-18T01:23:31.414717+0000 mon.a (mon.0) 973 : audit [INF] from='client.26920 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:32 smithi105 bash[19734]: cluster 2024-02-18T01:23:31.414813+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T01:23:32.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:32 smithi155 bash[20805]: audit 2024-02-18T01:23:31.414265+0000 mon.a (mon.0) 970 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:32 smithi155 bash[20805]: audit 2024-02-18T01:23:31.414400+0000 mon.a (mon.0) 971 : audit [INF] from='client.26881 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:32 smithi155 bash[20805]: audit 2024-02-18T01:23:31.414595+0000 mon.a (mon.0) 972 : audit [INF] from='client.26882 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:32 smithi155 bash[20805]: audit 2024-02-18T01:23:31.414717+0000 mon.a (mon.0) 973 : audit [INF] from='client.26920 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi105-61914-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:32.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:32 smithi155 bash[20805]: cluster 2024-02-18T01:23:31.414813+0000 mon.a (mon.0) 974 : cluster [DBG] osdmap e87: 8 total, 8 up, 8 in 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:32.305435+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: cluster 2024-02-18T01:23:32.522649+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:32.542593+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:32.543741+0000 mon.c (mon.2) 125 : audit [INF] from='client.? v1:172.21.15.105:0/1799256522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:32.564818+0000 mon.a (mon.0) 976 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:32.571437+0000 mon.a (mon.0) 977 : audit [INF] from='client.29603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: cluster 2024-02-18T01:23:32.933086+0000 mgr.y (mgr.24475) 255 : cluster [DBG] pgmap v208: 552 pgs: 64 unknown, 488 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:33.306816+0000 mon.c (mon.2) 126 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:33.524464+0000 mon.a (mon.0) 978 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: audit 2024-02-18T01:23:33.524595+0000 mon.a (mon.0) 979 : audit [INF] from='client.29603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[27669]: cluster 2024-02-18T01:23:33.524745+0000 mon.a (mon.0) 980 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:32.305435+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: cluster 2024-02-18T01:23:32.522649+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:32.542593+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:32.543741+0000 mon.c (mon.2) 125 : audit [INF] from='client.? v1:172.21.15.105:0/1799256522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:32.564818+0000 mon.a (mon.0) 976 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:32.571437+0000 mon.a (mon.0) 977 : audit [INF] from='client.29603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: cluster 2024-02-18T01:23:32.933086+0000 mgr.y (mgr.24475) 255 : cluster [DBG] pgmap v208: 552 pgs: 64 unknown, 488 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:33.306816+0000 mon.c (mon.2) 126 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:33.524464+0000 mon.a (mon.0) 978 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: audit 2024-02-18T01:23:33.524595+0000 mon.a (mon.0) 979 : audit [INF] from='client.29603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:33 smithi105 bash[19734]: cluster 2024-02-18T01:23:33.524745+0000 mon.a (mon.0) 980 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T01:23:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:32.305435+0000 mon.c (mon.2) 124 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: cluster 2024-02-18T01:23:32.522649+0000 mon.a (mon.0) 975 : cluster [DBG] osdmap e88: 8 total, 8 up, 8 in 2024-02-18T01:23:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:32.542593+0000 mon.b (mon.1) 81 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:32.543741+0000 mon.c (mon.2) 125 : audit [INF] from='client.? v1:172.21.15.105:0/1799256522' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:32.564818+0000 mon.a (mon.0) 976 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:32.571437+0000 mon.a (mon.0) 977 : audit [INF] from='client.29603 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: cluster 2024-02-18T01:23:32.933086+0000 mgr.y (mgr.24475) 255 : cluster [DBG] pgmap v208: 552 pgs: 64 unknown, 488 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 4.5 KiB/s wr, 8 op/s 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:33.306816+0000 mon.c (mon.2) 126 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:33.524464+0000 mon.a (mon.0) 978 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: audit 2024-02-18T01:23:33.524595+0000 mon.a (mon.0) 979 : audit [INF] from='client.29603 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi105-61649-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:33.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:33 smithi155 bash[20805]: cluster 2024-02-18T01:23:33.524745+0000 mon.a (mon.0) 980 : cluster [DBG] osdmap e89: 8 total, 8 up, 8 in 2024-02-18T01:23:34.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:33.585015+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:33.585782+0000 mon.a (mon.0) 981 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:34.309505+0000 mon.c (mon.2) 127 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: cluster 2024-02-18T01:23:34.521125+0000 mon.a (mon.0) 982 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:34.534591+0000 mon.a (mon.0) 983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: cluster 2024-02-18T01:23:34.534742+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:34.538702+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:34.550568+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:34.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[19734]: audit 2024-02-18T01:23:34.551560+0000 mon.a (mon.0) 986 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:34.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:33.585015+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:33.585782+0000 mon.a (mon.0) 981 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.995 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:34.309505+0000 mon.c (mon.2) 127 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:34.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: cluster 2024-02-18T01:23:34.521125+0000 mon.a (mon.0) 982 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:23:34.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:34.534591+0000 mon.a (mon.0) 983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:34.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: cluster 2024-02-18T01:23:34.534742+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T01:23:34.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:34.538702+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:34.996 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:34.550568+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:34.997 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:34 smithi105 bash[27669]: audit 2024-02-18T01:23:34.551560+0000 mon.a (mon.0) 986 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:34.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:33.585015+0000 mon.b (mon.1) 82 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:33.585782+0000 mon.a (mon.0) 981 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:34.309505+0000 mon.c (mon.2) 127 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: cluster 2024-02-18T01:23:34.521125+0000 mon.a (mon.0) 982 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:34.534591+0000 mon.a (mon.0) 983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: cluster 2024-02-18T01:23:34.534742+0000 mon.a (mon.0) 984 : cluster [DBG] osdmap e90: 8 total, 8 up, 8 in 2024-02-18T01:23:34.998 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:34.538702+0000 mon.a (mon.0) 985 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:34.999 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:34.550568+0000 mon.b (mon.1) 83 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:34.999 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:34 smithi155 bash[20805]: audit 2024-02-18T01:23:34.551560+0000 mon.a (mon.0) 986 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:35.583 INFO:tasks.workunit.client.0.smithi105.stdout: "zonegroup_name": "default" 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "task_status": {} 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: attempt 1 of 20 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: failed to deregister: 2024-02-18T01:23:35.584 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "epoch": 21, 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "modified": "2024-02-18T01:23:32.932612+0000", 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "services": { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "laundry": { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "daemons": { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "summary": "", 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "close-test-pid62154": { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_epoch": 20, 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_stamp": "2024-02-18T01:23:30.468851+0000", 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "gid": 28759, 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "addr": "172.21.15.105:0/1777770812", 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "metadata": { 2024-02-18T01:23:35.585 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro": "ubuntu", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_description": "Ubuntu 18.04.6 LTS", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_version": "18.04", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "foo": "bar", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "hostname": "smithi105", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_total_kb": "32823852", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "os": "Linux", 2024-02-18T01:23:35.586 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "this": "that" 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "task_status": {} 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "rgw": { 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "daemons": { 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "summary": "", 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "24398": { 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_epoch": 13, 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "start_stamp": "2024-02-18T01:17:17.946941+0000", 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "gid": 24398, 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "addr": "172.21.15.105:0/2475373584", 2024-02-18T01:23:35.587 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "metadata": { 2024-02-18T01:23:35.588 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "arch": "x86_64", 2024-02-18T01:23:35.588 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_release": "pacific", 2024-02-18T01:23:35.588 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version": "ceph version 16.2.14-548-g81bd20d6 (81bd20d634209c7cb82c18be12b4b5a05643ebf1) pacific (stable)", 2024-02-18T01:23:35.588 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "ceph_version_short": "16.2.14-548-g81bd20d6", 2024-02-18T01:23:35.588 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "container_hostname": "smithi105", 2024-02-18T01:23:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: cluster 2024-02-18T01:23:34.933779+0000 mgr.y (mgr.24475) 256 : cluster [DBG] pgmap v211: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.310673+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.566032+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? v1:172.21.15.105:0/2490207787' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.688712+0000 mon.a (mon.0) 987 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.689048+0000 mon.a (mon.0) 988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: cluster 2024-02-18T01:23:35.689230+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.704569+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.105:0/1636371945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.717703+0000 mon.a (mon.0) 990 : audit [INF] from='client.32584 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.720388+0000 mon.c (mon.2) 130 : audit [INF] from='client.? v1:172.21.15.105:0/3179229405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:35.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:35 smithi155 bash[20805]: audit 2024-02-18T01:23:35.721647+0000 mon.a (mon.0) 991 : audit [INF] from='client.32360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: cluster 2024-02-18T01:23:34.933779+0000 mgr.y (mgr.24475) 256 : cluster [DBG] pgmap v211: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.310673+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.566032+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? v1:172.21.15.105:0/2490207787' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.688712+0000 mon.a (mon.0) 987 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.689048+0000 mon.a (mon.0) 988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: cluster 2024-02-18T01:23:35.689230+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.704569+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.105:0/1636371945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.717703+0000 mon.a (mon.0) 990 : audit [INF] from='client.32584 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.720388+0000 mon.c (mon.2) 130 : audit [INF] from='client.? v1:172.21.15.105:0/3179229405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[27669]: audit 2024-02-18T01:23:35.721647+0000 mon.a (mon.0) 991 : audit [INF] from='client.32360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: cluster 2024-02-18T01:23:34.933779+0000 mgr.y (mgr.24475) 256 : cluster [DBG] pgmap v211: 488 pgs: 64 unknown, 424 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 5 op/s 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.310673+0000 mon.c (mon.2) 128 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.566032+0000 mon.c (mon.2) 129 : audit [DBG] from='client.? v1:172.21.15.105:0/2490207787' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.688712+0000 mon.a (mon.0) 987 : audit [INF] from='client.? v1:172.21.15.105:0/3383185212' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-62142-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.689048+0000 mon.a (mon.0) 988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: cluster 2024-02-18T01:23:35.689230+0000 mon.a (mon.0) 989 : cluster [DBG] osdmap e91: 8 total, 8 up, 8 in 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.704569+0000 mon.b (mon.1) 84 : audit [INF] from='client.? v1:172.21.15.105:0/1636371945' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.717703+0000 mon.a (mon.0) 990 : audit [INF] from='client.32584 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.720388+0000 mon.c (mon.2) 130 : audit [INF] from='client.? v1:172.21.15.105:0/3179229405' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:35 smithi105 bash[19734]: audit 2024-02-18T01:23:35.721647+0000 mon.a (mon.0) 991 : audit [INF] from='client.32360 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:36.714 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "container_image": "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:7fa249cc01918 api_c_write_operations: Running main() from gmock_main.cc 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (13774 ms) 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (5421 ms) 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (3339 ms) 2024-02-18T01:23:36.715 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (3310 ms) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (2222 ms) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (5337 ms) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (3145 ms) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (36548 ms total) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (36548 ms total) 2024-02-18T01:23:36.716 INFO:tasks.workunit.client.0.smithi105.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-02-18T01:23:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:36] "GET /metrics HTTP/1.1" 200 207028 "" "Prometheus/2.33.4" 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:35.569404+0000 mgr.y (mgr.24475) 257 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:35.828417+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.313662+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.340271+0000 mon.a (mon.0) 992 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.501368+0000 mon.a (mon.0) 993 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.693735+0000 mon.a (mon.0) 994 : audit [INF] from='client.32584 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.693822+0000 mon.a (mon.0) 995 : audit [INF] from='client.32360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.589 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.694007+0000 mon.a (mon.0) 996 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:37.590 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: cluster 2024-02-18T01:23:36.694053+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T01:23:37.590 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.706058+0000 mon.b (mon.1) 86 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.590 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.712380+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.105:0/3754825022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:37.590 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[20805]: audit 2024-02-18T01:23:36.713094+0000 mon.a (mon.0) 998 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:35.569404+0000 mgr.y (mgr.24475) 257 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:35.828417+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.313662+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.340271+0000 mon.a (mon.0) 992 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.501368+0000 mon.a (mon.0) 993 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.693735+0000 mon.a (mon.0) 994 : audit [INF] from='client.32584 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.693822+0000 mon.a (mon.0) 995 : audit [INF] from='client.32360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.694007+0000 mon.a (mon.0) 996 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: cluster 2024-02-18T01:23:36.694053+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.706058+0000 mon.b (mon.1) 86 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.712380+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.105:0/3754825022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[19734]: audit 2024-02-18T01:23:36.713094+0000 mon.a (mon.0) 998 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:35.569404+0000 mgr.y (mgr.24475) 257 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:35.828417+0000 mon.b (mon.1) 85 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.313662+0000 mon.c (mon.2) 131 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.340271+0000 mon.a (mon.0) 992 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.501368+0000 mon.a (mon.0) 993 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.693735+0000 mon.a (mon.0) 994 : audit [INF] from='client.32584 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.693822+0000 mon.a (mon.0) 995 : audit [INF] from='client.32360 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi105-61649-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.694007+0000 mon.a (mon.0) 996 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: cluster 2024-02-18T01:23:36.694053+0000 mon.a (mon.0) 997 : cluster [DBG] osdmap e92: 8 total, 8 up, 8 in 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.706058+0000 mon.b (mon.1) 86 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.712380+0000 mon.b (mon.1) 87 : audit [INF] from='client.? v1:172.21.15.105:0/3754825022' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:37 smithi105 bash[27669]: audit 2024-02-18T01:23:36.713094+0000 mon.a (mon.0) 998 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]: dispatch 2024-02-18T01:23:37.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:36.816611+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: cluster 2024-02-18T01:23:36.934451+0000 mgr.y (mgr.24475) 258 : cluster [DBG] pgmap v214: 488 pgs: 128 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.239010+0000 mon.a (mon.0) 999 : audit [INF] from='client.33559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.251306+0000 mon.a (mon.0) 1000 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.254706+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.255078+0000 mon.a (mon.0) 1001 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.255736+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.256086+0000 mon.a (mon.0) 1002 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.318744+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.701881+0000 mon.a (mon.0) 1003 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.702088+0000 mon.a (mon.0) 1004 : audit [INF] from='client.33559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.702221+0000 mon.a (mon.0) 1005 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:38.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: cluster 2024-02-18T01:23:37.702281+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.714964+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[19734]: audit 2024-02-18T01:23:37.718222+0000 mon.a (mon.0) 1007 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:36.816611+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: cluster 2024-02-18T01:23:36.934451+0000 mgr.y (mgr.24475) 258 : cluster [DBG] pgmap v214: 488 pgs: 128 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.239010+0000 mon.a (mon.0) 999 : audit [INF] from='client.33559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.251306+0000 mon.a (mon.0) 1000 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.254706+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.255078+0000 mon.a (mon.0) 1001 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.255736+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.256086+0000 mon.a (mon.0) 1002 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.318744+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.701881+0000 mon.a (mon.0) 1003 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.702088+0000 mon.a (mon.0) 1004 : audit [INF] from='client.33559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.702221+0000 mon.a (mon.0) 1005 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: cluster 2024-02-18T01:23:37.702281+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.714964+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:38 smithi105 bash[27669]: audit 2024-02-18T01:23:37.718222+0000 mon.a (mon.0) 1007 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:36.816611+0000 mon.b (mon.1) 88 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: cluster 2024-02-18T01:23:36.934451+0000 mgr.y (mgr.24475) 258 : cluster [DBG] pgmap v214: 488 pgs: 128 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.239010+0000 mon.a (mon.0) 999 : audit [INF] from='client.33559 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.251306+0000 mon.a (mon.0) 1000 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.254706+0000 mon.b (mon.1) 89 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.255078+0000 mon.a (mon.0) 1001 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.255736+0000 mon.b (mon.1) 90 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.256086+0000 mon.a (mon.0) 1002 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.318744+0000 mon.c (mon.2) 132 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.701881+0000 mon.a (mon.0) 1003 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-7"}]': finished 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.702088+0000 mon.a (mon.0) 1004 : audit [INF] from='client.33559 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.702221+0000 mon.a (mon.0) 1005 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: cluster 2024-02-18T01:23:37.702281+0000 mon.a (mon.0) 1006 : cluster [DBG] osdmap e93: 8 total, 8 up, 8 in 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.714964+0000 mon.b (mon.1) 91 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:38.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:38 smithi155 bash[20805]: audit 2024-02-18T01:23:37.718222+0000 mon.a (mon.0) 1007 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: audit 2024-02-18T01:23:37.587069+0000 mgr.y (mgr.24475) 259 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: audit 2024-02-18T01:23:38.320448+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: cluster 2024-02-18T01:23:38.935058+0000 mgr.y (mgr.24475) 260 : cluster [DBG] pgmap v217: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: cluster 2024-02-18T01:23:39.217987+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: audit 2024-02-18T01:23:39.227039+0000 mon.c (mon.2) 134 : audit [INF] from='client.? v1:172.21.15.105:0/826246527' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: audit 2024-02-18T01:23:39.325358+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[19734]: audit 2024-02-18T01:23:39.746355+0000 mon.a (mon.0) 1009 : audit [INF] from='client.34937 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: audit 2024-02-18T01:23:37.587069+0000 mgr.y (mgr.24475) 259 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: audit 2024-02-18T01:23:38.320448+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: cluster 2024-02-18T01:23:38.935058+0000 mgr.y (mgr.24475) 260 : cluster [DBG] pgmap v217: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: cluster 2024-02-18T01:23:39.217987+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: audit 2024-02-18T01:23:39.227039+0000 mon.c (mon.2) 134 : audit [INF] from='client.? v1:172.21.15.105:0/826246527' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: audit 2024-02-18T01:23:39.325358+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:40 smithi105 bash[27669]: audit 2024-02-18T01:23:39.746355+0000 mon.a (mon.0) 1009 : audit [INF] from='client.34937 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: audit 2024-02-18T01:23:37.587069+0000 mgr.y (mgr.24475) 259 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: audit 2024-02-18T01:23:38.320448+0000 mon.c (mon.2) 133 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: cluster 2024-02-18T01:23:38.935058+0000 mgr.y (mgr.24475) 260 : cluster [DBG] pgmap v217: 424 pgs: 32 unknown, 392 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: cluster 2024-02-18T01:23:39.217987+0000 mon.a (mon.0) 1008 : cluster [DBG] osdmap e94: 8 total, 8 up, 8 in 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: audit 2024-02-18T01:23:39.227039+0000 mon.c (mon.2) 134 : audit [INF] from='client.? v1:172.21.15.105:0/826246527' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: audit 2024-02-18T01:23:39.325358+0000 mon.c (mon.2) 135 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:40 smithi155 bash[20805]: audit 2024-02-18T01:23:39.746355+0000 mon.a (mon.0) 1009 : audit [INF] from='client.34937 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout:69e2bcc743303b0da933d04d152b2397edb19ec992e48553ab9", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "cpu": "Intel(R) Xeon(R) CPU E5-1620 v4 @ 3.50GHz", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro": "centos", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_description": "CentOS Stream 8", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "distro_version": "8", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "frontend_config#0": "beast port=80", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "frontend_type#0": "beast", 2024-02-18T01:23:40.807 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "hostname": "smithi105", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "id": "foo.a", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_description": "#224-Ubuntu SMP Mon Jun 19 13:30:12 UTC 2023", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "kernel_version": "4.15.0-213-generic", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_swap_kb": "2097148", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "mem_total_kb": "32823852", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "num_handles": "1", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "os": "Linux", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "pid": "7", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "realm_id": "", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "realm_name": "", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zone_id": "359ef169-be4f-46cd-ad28-f5ca4f52f983", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zone_name": "default", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zonegroup_id": "fc6cf9f5-802c-4a69-9859-d03009cee767", 2024-02-18T01:23:40.808 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "zonegroup_name": "default" 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: }, 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: "task_status": {} 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: } 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: attempt 2 of 20 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (15493 ms) 2024-02-18T01:23:40.809 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (40615 ms total) 2024-02-18T01:23:40.810 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: 2024-02-18T01:23:40.810 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [----------] Global test environment tear-down 2024-02-18T01:23:40.810 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (40616 ms total) 2024-02-18T01:23:40.810 INFO:tasks.workunit.client.0.smithi105.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:39.811296+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.267421+0000 mon.a (mon.0) 1010 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.286354+0000 mon.a (mon.0) 1011 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.286569+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34937 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: cluster 2024-02-18T01:23:40.286726+0000 mon.a (mon.0) 1013 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.293352+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.105:0/2541666240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.294150+0000 mon.a (mon.0) 1014 : audit [INF] from='client.36721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.307126+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.308668+0000 mon.a (mon.0) 1015 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.338241+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.785543+0000 mon.c (mon.2) 138 : audit [DBG] from='client.? v1:172.21.15.105:0/1514683907' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: audit 2024-02-18T01:23:40.786433+0000 mgr.y (mgr.24475) 261 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[19734]: cluster 2024-02-18T01:23:40.936189+0000 mgr.y (mgr.24475) 262 : cluster [DBG] pgmap v219: 464 pgs: 104 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:39.811296+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.267421+0000 mon.a (mon.0) 1010 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.286354+0000 mon.a (mon.0) 1011 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.286569+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34937 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:41.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: cluster 2024-02-18T01:23:40.286726+0000 mon.a (mon.0) 1013 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.293352+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.105:0/2541666240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.294150+0000 mon.a (mon.0) 1014 : audit [INF] from='client.36721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.307126+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.308668+0000 mon.a (mon.0) 1015 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.338241+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.785543+0000 mon.c (mon.2) 138 : audit [DBG] from='client.? v1:172.21.15.105:0/1514683907' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: audit 2024-02-18T01:23:40.786433+0000 mgr.y (mgr.24475) 261 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:41 smithi105 bash[27669]: cluster 2024-02-18T01:23:40.936189+0000 mgr.y (mgr.24475) 262 : cluster [DBG] pgmap v219: 464 pgs: 104 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:39.811296+0000 mon.c (mon.2) 136 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.267421+0000 mon.a (mon.0) 1010 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]: dispatch 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.286354+0000 mon.a (mon.0) 1011 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi105-61891-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.286569+0000 mon.a (mon.0) 1012 : audit [INF] from='client.34937 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi105-61649-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: cluster 2024-02-18T01:23:40.286726+0000 mon.a (mon.0) 1013 : cluster [DBG] osdmap e95: 8 total, 8 up, 8 in 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.293352+0000 mon.b (mon.1) 92 : audit [INF] from='client.? v1:172.21.15.105:0/2541666240' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.294150+0000 mon.a (mon.0) 1014 : audit [INF] from='client.36721 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.307126+0000 mon.b (mon.1) 93 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.308668+0000 mon.a (mon.0) 1015 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.338241+0000 mon.c (mon.2) 137 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.785543+0000 mon.c (mon.2) 138 : audit [DBG] from='client.? v1:172.21.15.105:0/1514683907' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: audit 2024-02-18T01:23:40.786433+0000 mgr.y (mgr.24475) 261 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-02-18T01:23:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[20805]: cluster 2024-02-18T01:23:40.936189+0000 mgr.y (mgr.24475) 262 : cluster [DBG] pgmap v219: 464 pgs: 104 unknown, 360 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:23:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.286161+0000 mon.a (mon.0) 1016 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]': finished 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.286266+0000 mon.a (mon.0) 1017 : audit [INF] from='client.36721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.286430+0000 mon.a (mon.0) 1018 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: cluster 2024-02-18T01:23:41.286547+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.292588+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.305541+0000 mon.c (mon.2) 139 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.306648+0000 mon.a (mon.0) 1021 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.339158+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.341642+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:41.347435+0000 mon.c (mon.2) 140 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: cluster 2024-02-18T01:23:42.283655+0000 mon.a (mon.0) 1023 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:42.291563+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:42.291672+0000 mon.a (mon.0) 1025 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:42.291835+0000 mon.a (mon.0) 1026 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: cluster 2024-02-18T01:23:42.291977+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:42.294692+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[19734]: audit 2024-02-18T01:23:42.296191+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.286161+0000 mon.a (mon.0) 1016 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.286266+0000 mon.a (mon.0) 1017 : audit [INF] from='client.36721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.286430+0000 mon.a (mon.0) 1018 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: cluster 2024-02-18T01:23:41.286547+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.292588+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.305541+0000 mon.c (mon.2) 139 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.306648+0000 mon.a (mon.0) 1021 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.339158+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.341642+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:41.347435+0000 mon.c (mon.2) 140 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: cluster 2024-02-18T01:23:42.283655+0000 mon.a (mon.0) 1023 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:42.291563+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:42.291672+0000 mon.a (mon.0) 1025 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:42.291835+0000 mon.a (mon.0) 1026 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: cluster 2024-02-18T01:23:42.291977+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T01:23:42.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:42.294692+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:42.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:42 smithi105 bash[27669]: audit 2024-02-18T01:23:42.296191+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.286161+0000 mon.a (mon.0) 1016 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4"}]': finished 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.286266+0000 mon.a (mon.0) 1017 : audit [INF] from='client.36721 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi105-61657-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.286430+0000 mon.a (mon.0) 1018 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: cluster 2024-02-18T01:23:41.286547+0000 mon.a (mon.0) 1019 : cluster [DBG] osdmap e96: 8 total, 8 up, 8 in 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.292588+0000 mon.a (mon.0) 1020 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.305541+0000 mon.c (mon.2) 139 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.306648+0000 mon.a (mon.0) 1021 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.339158+0000 mon.b (mon.1) 94 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.341642+0000 mon.a (mon.0) 1022 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:41.347435+0000 mon.c (mon.2) 140 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: cluster 2024-02-18T01:23:42.283655+0000 mon.a (mon.0) 1023 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:42.291563+0000 mon.a (mon.0) 1024 : audit [INF] from='client.? v1:172.21.15.105:0/3044536421' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:42.291672+0000 mon.a (mon.0) 1025 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6"}]': finished 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:42.291835+0000 mon.a (mon.0) 1026 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: cluster 2024-02-18T01:23:42.291977+0000 mon.a (mon.0) 1027 : cluster [DBG] osdmap e97: 8 total, 8 up, 8 in 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:42.294692+0000 mon.b (mon.1) 95 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:42.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:42 smithi155 bash[20805]: audit 2024-02-18T01:23:42.296191+0000 mon.a (mon.0) 1028 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[27669]: audit 2024-02-18T01:23:42.297933+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[27669]: audit 2024-02-18T01:23:42.299691+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.105:0/4001264192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[27669]: audit 2024-02-18T01:23:42.312246+0000 mon.a (mon.0) 1029 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[27669]: audit 2024-02-18T01:23:42.312685+0000 mon.a (mon.0) 1030 : audit [INF] from='client.38575 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[27669]: audit 2024-02-18T01:23:42.355904+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[19734]: audit 2024-02-18T01:23:42.297933+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[19734]: audit 2024-02-18T01:23:42.299691+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.105:0/4001264192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[19734]: audit 2024-02-18T01:23:42.312246+0000 mon.a (mon.0) 1029 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[19734]: audit 2024-02-18T01:23:42.312685+0000 mon.a (mon.0) 1030 : audit [INF] from='client.38575 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:43 smithi105 bash[19734]: audit 2024-02-18T01:23:42.355904+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:43 smithi155 bash[20805]: audit 2024-02-18T01:23:42.297933+0000 mon.c (mon.2) 141 : audit [INF] from='client.? v1:172.21.15.105:0/2692890775' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:43 smithi155 bash[20805]: audit 2024-02-18T01:23:42.299691+0000 mon.b (mon.1) 96 : audit [INF] from='client.? v1:172.21.15.105:0/4001264192' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:43 smithi155 bash[20805]: audit 2024-02-18T01:23:42.312246+0000 mon.a (mon.0) 1029 : audit [INF] from='client.25211 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:43 smithi155 bash[20805]: audit 2024-02-18T01:23:42.312685+0000 mon.a (mon.0) 1030 : audit [INF] from='client.38575 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:43 smithi155 bash[20805]: audit 2024-02-18T01:23:42.355904+0000 mon.c (mon.2) 142 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout:i_watch_notify_pp: flushed 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3011 ms) 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: trying... 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337728 notify_id 369367187466 notifier_gid 25211 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: timed out 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: flushing 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: flushed 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3008 ms) 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-02-18T01:23:44.415 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: List watches 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify2 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337728 notify_id 382252089355 notifier_gid 25211 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify2 done 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: watch_check 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: unwatch2 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: flushing 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: done 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3046 ms) 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: List watches 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify2 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: handle_notify cookie 94534337337728 notify_id 395136991244 notifier_gid 25211 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: notify2 done 2024-02-18T01:23:44.416 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: watch_check 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: unwatch2 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: flushing 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: done 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3053 ms) 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (17120 ms total) 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (44325 ms total) 2024-02-18T01:23:44.417 INFO:tasks.workunit.client.0.smithi105.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-02-18T01:23:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: cluster 2024-02-18T01:23:42.938436+0000 mgr.y (mgr.24475) 263 : cluster [DBG] pgmap v222: 464 pgs: 28 creating+peering, 1 active, 23 creating+activating, 4 unknown, 408 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.4 KiB/s wr, 4 op/s 2024-02-18T01:23:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.360938+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.396179+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.396268+0000 mon.a (mon.0) 1032 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.396347+0000 mon.a (mon.0) 1033 : audit [INF] from='client.38575 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: cluster 2024-02-18T01:23:43.396637+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.408027+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.411231+0000 mon.a (mon.0) 1035 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.432269+0000 mon.c (mon.2) 144 : audit [INF] from='client.? v1:172.21.15.105:0/1370921220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:43.433974+0000 mon.a (mon.0) 1036 : audit [INF] from='client.38870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: cluster 2024-02-18T01:23:43.442131+0000 mon.a (mon.0) 1037 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:44.362398+0000 mon.c (mon.2) 145 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: cluster 2024-02-18T01:23:44.390361+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:44.396024+0000 mon.a (mon.0) 1039 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]': finished 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: audit 2024-02-18T01:23:44.396219+0000 mon.a (mon.0) 1040 : audit [INF] from='client.38870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[19734]: cluster 2024-02-18T01:23:44.396364+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: cluster 2024-02-18T01:23:42.938436+0000 mgr.y (mgr.24475) 263 : cluster [DBG] pgmap v222: 464 pgs: 28 creating+peering, 1 active, 23 creating+activating, 4 unknown, 408 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.4 KiB/s wr, 4 op/s 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.360938+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.396179+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.396268+0000 mon.a (mon.0) 1032 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.396347+0000 mon.a (mon.0) 1033 : audit [INF] from='client.38575 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: cluster 2024-02-18T01:23:43.396637+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.408027+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.411231+0000 mon.a (mon.0) 1035 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.432269+0000 mon.c (mon.2) 144 : audit [INF] from='client.? v1:172.21.15.105:0/1370921220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:43.433974+0000 mon.a (mon.0) 1036 : audit [INF] from='client.38870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: cluster 2024-02-18T01:23:43.442131+0000 mon.a (mon.0) 1037 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:44.362398+0000 mon.c (mon.2) 145 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: cluster 2024-02-18T01:23:44.390361+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:44.396024+0000 mon.a (mon.0) 1039 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]': finished 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: audit 2024-02-18T01:23:44.396219+0000 mon.a (mon.0) 1040 : audit [INF] from='client.38870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:44 smithi105 bash[27669]: cluster 2024-02-18T01:23:44.396364+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: cluster 2024-02-18T01:23:42.938436+0000 mgr.y (mgr.24475) 263 : cluster [DBG] pgmap v222: 464 pgs: 28 creating+peering, 1 active, 23 creating+activating, 4 unknown, 408 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.9 KiB/s rd, 2.4 KiB/s wr, 4 op/s 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.360938+0000 mon.c (mon.2) 143 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.396179+0000 mon.a (mon.0) 1031 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.396268+0000 mon.a (mon.0) 1032 : audit [INF] from='client.25211 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi105-61976-6", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.396347+0000 mon.a (mon.0) 1033 : audit [INF] from='client.38575 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi105-61649-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: cluster 2024-02-18T01:23:43.396637+0000 mon.a (mon.0) 1034 : cluster [DBG] osdmap e98: 8 total, 8 up, 8 in 2024-02-18T01:23:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.408027+0000 mon.b (mon.1) 97 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.411231+0000 mon.a (mon.0) 1035 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]: dispatch 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.432269+0000 mon.c (mon.2) 144 : audit [INF] from='client.? v1:172.21.15.105:0/1370921220' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:43.433974+0000 mon.a (mon.0) 1036 : audit [INF] from='client.38870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: cluster 2024-02-18T01:23:43.442131+0000 mon.a (mon.0) 1037 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:44.362398+0000 mon.c (mon.2) 145 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: cluster 2024-02-18T01:23:44.390361+0000 mon.a (mon.0) 1038 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:44.396024+0000 mon.a (mon.0) 1039 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-9", "mode": "writeback"}]': finished 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: audit 2024-02-18T01:23:44.396219+0000 mon.a (mon.0) 1040 : audit [INF] from='client.38870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi105-61657-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:44.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:44 smithi155 bash[20805]: cluster 2024-02-18T01:23:44.396364+0000 mon.a (mon.0) 1041 : cluster [DBG] osdmap e99: 8 total, 8 up, 8 in 2024-02-18T01:23:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.565727+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.790183+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.790380+0000 mgr.y (mgr.24475) 264 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.793429+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.794071+0000 mgr.y (mgr.24475) 265 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.801365+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.801844+0000 mgr.y (mgr.24475) 266 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.804524+0000 mon.c (mon.2) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.804702+0000 mgr.y (mgr.24475) 267 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.805227+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.805439+0000 mgr.y (mgr.24475) 268 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.806569+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.806721+0000 mgr.y (mgr.24475) 269 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.808247+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.808437+0000 mgr.y (mgr.24475) 270 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.808885+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.809051+0000 mgr.y (mgr.24475) 271 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.809928+0000 mon.c (mon.2) 154 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.810132+0000 mgr.y (mgr.24475) 272 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.810824+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:44.811003+0000 mgr.y (mgr.24475) 273 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: cluster 2024-02-18T01:23:44.939128+0000 mgr.y (mgr.24475) 274 : cluster [DBG] pgmap v225: 428 pgs: 1 active, 23 creating+activating, 32 unknown, 372 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.005073+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.363725+0000 mon.c (mon.2) 156 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.400856+0000 mon.a (mon.0) 1043 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: cluster 2024-02-18T01:23:45.400931+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.406095+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.409104+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.421269+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.105:0/830013743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[27669]: audit 2024-02-18T01:23:45.421871+0000 mon.a (mon.0) 1046 : audit [INF] from='client.40718 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.565727+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.790183+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.790380+0000 mgr.y (mgr.24475) 264 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.793429+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.794071+0000 mgr.y (mgr.24475) 265 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.801365+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.801844+0000 mgr.y (mgr.24475) 266 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.804524+0000 mon.c (mon.2) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.804702+0000 mgr.y (mgr.24475) 267 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.805227+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.747 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.805439+0000 mgr.y (mgr.24475) 268 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.747 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.806569+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.747 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.806721+0000 mgr.y (mgr.24475) 269 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.747 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.808247+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.808437+0000 mgr.y (mgr.24475) 270 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.808885+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.809051+0000 mgr.y (mgr.24475) 271 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.809928+0000 mon.c (mon.2) 154 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.810132+0000 mgr.y (mgr.24475) 272 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.810824+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:44.811003+0000 mgr.y (mgr.24475) 273 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: cluster 2024-02-18T01:23:44.939128+0000 mgr.y (mgr.24475) 274 : cluster [DBG] pgmap v225: 428 pgs: 1 active, 23 creating+activating, 32 unknown, 372 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.005073+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.363725+0000 mon.c (mon.2) 156 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.400856+0000 mon.a (mon.0) 1043 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: cluster 2024-02-18T01:23:45.400931+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.406095+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.409104+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.421269+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.105:0/830013743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:45 smithi105 bash[19734]: audit 2024-02-18T01:23:45.421871+0000 mon.a (mon.0) 1046 : audit [INF] from='client.40718 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.565727+0000 mon.b (mon.1) 98 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.790183+0000 mon.c (mon.2) 146 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.790380+0000 mgr.y (mgr.24475) 264 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.0"}]: dispatch 2024-02-18T01:23:45.749 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.793429+0000 mon.c (mon.2) 147 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.794071+0000 mgr.y (mgr.24475) 265 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.1"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.801365+0000 mon.c (mon.2) 148 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.801844+0000 mgr.y (mgr.24475) 266 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.2"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.804524+0000 mon.c (mon.2) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.804702+0000 mgr.y (mgr.24475) 267 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.3"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.805227+0000 mon.c (mon.2) 150 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.805439+0000 mgr.y (mgr.24475) 268 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.4"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.806569+0000 mon.c (mon.2) 151 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.806721+0000 mgr.y (mgr.24475) 269 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.5"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.808247+0000 mon.c (mon.2) 152 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.808437+0000 mgr.y (mgr.24475) 270 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.6"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.808885+0000 mon.c (mon.2) 153 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.809051+0000 mgr.y (mgr.24475) 271 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.7"}]: dispatch 2024-02-18T01:23:45.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.809928+0000 mon.c (mon.2) 154 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.810132+0000 mgr.y (mgr.24475) 272 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.8"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.810824+0000 mon.c (mon.2) 155 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:44.811003+0000 mgr.y (mgr.24475) 273 : audit [DBG] from='mon.? v1:172.21.15.105:0/1339595748' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "18.9"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: cluster 2024-02-18T01:23:44.939128+0000 mgr.y (mgr.24475) 274 : cluster [DBG] pgmap v225: 428 pgs: 1 active, 23 creating+activating, 32 unknown, 372 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.005073+0000 mon.a (mon.0) 1042 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.363725+0000 mon.c (mon.2) 156 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.400856+0000 mon.a (mon.0) 1043 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: cluster 2024-02-18T01:23:45.400931+0000 mon.a (mon.0) 1044 : cluster [DBG] osdmap e100: 8 total, 8 up, 8 in 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.406095+0000 mon.b (mon.1) 99 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.409104+0000 mon.a (mon.0) 1045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.421269+0000 mon.c (mon.2) 157 : audit [INF] from='client.? v1:172.21.15.105:0/830013743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:45.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:45 smithi155 bash[20805]: audit 2024-02-18T01:23:45.421871+0000 mon.a (mon.0) 1046 : audit [INF] from='client.40718 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.365231+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: cluster 2024-02-18T01:23:46.522657+0000 mon.a (mon.0) 1047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.532700+0000 mon.a (mon.0) 1048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.534249+0000 mon.a (mon.0) 1049 : audit [INF] from='client.40718 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: cluster 2024-02-18T01:23:46.534454+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.547594+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.558037+0000 mon.a (mon.0) 1051 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.963 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[19734]: audit 2024-02-18T01:23:46.570130+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:46] "GET /metrics HTTP/1.1" 200 207028 "" "Prometheus/2.33.4" 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.365231+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: cluster 2024-02-18T01:23:46.522657+0000 mon.a (mon.0) 1047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.532700+0000 mon.a (mon.0) 1048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.534249+0000 mon.a (mon.0) 1049 : audit [INF] from='client.40718 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:46.964 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: cluster 2024-02-18T01:23:46.534454+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T01:23:46.965 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.547594+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.965 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.558037+0000 mon.a (mon.0) 1051 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:46.965 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:46 smithi105 bash[27669]: audit 2024-02-18T01:23:46.570130+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.365231+0000 mon.c (mon.2) 158 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: cluster 2024-02-18T01:23:46.522657+0000 mon.a (mon.0) 1047 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.532700+0000 mon.a (mon.0) 1048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-9"}]': finished 2024-02-18T01:23:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.534249+0000 mon.a (mon.0) 1049 : audit [INF] from='client.40718 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi105-61649-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: cluster 2024-02-18T01:23:46.534454+0000 mon.a (mon.0) 1050 : cluster [DBG] osdmap e101: 8 total, 8 up, 8 in 2024-02-18T01:23:47.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.547594+0000 mon.b (mon.1) 100 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:47.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.558037+0000 mon.a (mon.0) 1051 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:47.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:46 smithi155 bash[20805]: audit 2024-02-18T01:23:46.570130+0000 mon.a (mon.0) 1052 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:47.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: cluster 2024-02-18T01:23:46.836430+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: cluster 2024-02-18T01:23:46.843062+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: cluster 2024-02-18T01:23:46.939783+0000 mgr.y (mgr.24475) 275 : cluster [DBG] pgmap v228: 460 pgs: 96 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: audit 2024-02-18T01:23:47.366179+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: audit 2024-02-18T01:23:47.531718+0000 mon.a (mon.0) 1053 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: audit 2024-02-18T01:23:47.531888+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[19734]: cluster 2024-02-18T01:23:47.531992+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T01:23:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: cluster 2024-02-18T01:23:46.836430+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: cluster 2024-02-18T01:23:46.843062+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: cluster 2024-02-18T01:23:46.939783+0000 mgr.y (mgr.24475) 275 : cluster [DBG] pgmap v228: 460 pgs: 96 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: audit 2024-02-18T01:23:47.366179+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: audit 2024-02-18T01:23:47.531718+0000 mon.a (mon.0) 1053 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: audit 2024-02-18T01:23:47.531888+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:48 smithi105 bash[27669]: cluster 2024-02-18T01:23:47.531992+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: cluster 2024-02-18T01:23:46.836430+0000 osd.3 (osd.3) 7 : cluster [DBG] 18.1 deep-scrub starts 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: cluster 2024-02-18T01:23:46.843062+0000 osd.3 (osd.3) 8 : cluster [DBG] 18.1 deep-scrub ok 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: cluster 2024-02-18T01:23:46.939783+0000 mgr.y (mgr.24475) 275 : cluster [DBG] pgmap v228: 460 pgs: 96 unknown, 364 active+clean; 144 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: audit 2024-02-18T01:23:47.366179+0000 mon.c (mon.2) 159 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: audit 2024-02-18T01:23:47.531718+0000 mon.a (mon.0) 1053 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61858-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: audit 2024-02-18T01:23:47.531888+0000 mon.a (mon.0) 1054 : audit [INF] from='client.? v1:172.21.15.105:0/417117313' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi105-61657-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:48.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:48 smithi155 bash[20805]: cluster 2024-02-18T01:23:47.531992+0000 mon.a (mon.0) 1055 : cluster [DBG] osdmap e102: 8 total, 8 up, 8 in 2024-02-18T01:23:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:47.438468+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:47.441783+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:47.597959+0000 mgr.y (mgr.24475) 276 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:47.872172+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:47.973412+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:48.293487+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:49.998 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:48.298054+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:48.367024+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:48.602445+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:48.626107+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:48.626376+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: cluster 2024-02-18T01:23:48.940495+0000 mgr.y (mgr.24475) 277 : cluster [DBG] pgmap v231: 428 pgs: 64 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:49.187983+0000 mon.a (mon.0) 1057 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[19734]: audit 2024-02-18T01:23:49.188264+0000 mon.a (mon.0) 1058 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:47.438468+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:47.441783+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:47.597959+0000 mgr.y (mgr.24475) 276 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:47.872172+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:49.999 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:47.973412+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:48.293487+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:48.298054+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:48.367024+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:48.602445+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:48.626107+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:48.626376+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: cluster 2024-02-18T01:23:48.940495+0000 mgr.y (mgr.24475) 277 : cluster [DBG] pgmap v231: 428 pgs: 64 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:49.187983+0000 mon.a (mon.0) 1057 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.000 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:49 smithi105 bash[27669]: audit 2024-02-18T01:23:49.188264+0000 mon.a (mon.0) 1058 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:47.438468+0000 osd.2 (osd.2) 5 : cluster [DBG] 18.7 deep-scrub starts 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:47.441783+0000 osd.2 (osd.2) 6 : cluster [DBG] 18.7 deep-scrub ok 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:47.597959+0000 mgr.y (mgr.24475) 276 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:47.872172+0000 osd.3 (osd.3) 9 : cluster [DBG] 18.9 deep-scrub starts 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:47.973412+0000 osd.3 (osd.3) 10 : cluster [DBG] 18.9 deep-scrub ok 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:48.293487+0000 osd.0 (osd.0) 9 : cluster [DBG] 18.0 deep-scrub starts 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:48.298054+0000 osd.0 (osd.0) 10 : cluster [DBG] 18.0 deep-scrub ok 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:48.367024+0000 mon.c (mon.2) 160 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:48.602445+0000 mon.a (mon.0) 1056 : cluster [DBG] osdmap e103: 8 total, 8 up, 8 in 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:48.626107+0000 mon.b (mon.1) 101 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:48.626376+0000 mon.b (mon.1) 102 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:50.001 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: cluster 2024-02-18T01:23:48.940495+0000 mgr.y (mgr.24475) 277 : cluster [DBG] pgmap v231: 428 pgs: 64 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:50.002 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:49.187983+0000 mon.a (mon.0) 1057 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.002 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:49 smithi155 bash[20805]: audit 2024-02-18T01:23:49.188264+0000 mon.a (mon.0) 1058 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]: dispatch 2024-02-18T01:23:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.369364+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: cluster 2024-02-18T01:23:49.621751+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: cluster 2024-02-18T01:23:49.624938+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.724099+0000 mon.a (mon.0) 1059 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.724178+0000 mon.a (mon.0) 1060 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]': finished 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: cluster 2024-02-18T01:23:49.724260+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.739709+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.740091+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.105:0/4134174628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.740556+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.105:0/3476391272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.740852+0000 mon.a (mon.0) 1062 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.742305+0000 mon.a (mon.0) 1063 : audit [INF] from='client.44710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:49.742551+0000 mon.a (mon.0) 1064 : audit [INF] from='client.43870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: cluster 2024-02-18T01:23:50.295047+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: cluster 2024-02-18T01:23:50.298806+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[19734]: audit 2024-02-18T01:23:50.371231+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.369364+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: cluster 2024-02-18T01:23:49.621751+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: cluster 2024-02-18T01:23:49.624938+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.724099+0000 mon.a (mon.0) 1059 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.724178+0000 mon.a (mon.0) 1060 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]': finished 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: cluster 2024-02-18T01:23:49.724260+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.739709+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.740091+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.105:0/4134174628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.740556+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.105:0/3476391272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.740852+0000 mon.a (mon.0) 1062 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.742305+0000 mon.a (mon.0) 1063 : audit [INF] from='client.44710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:49.742551+0000 mon.a (mon.0) 1064 : audit [INF] from='client.43870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: cluster 2024-02-18T01:23:50.295047+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: cluster 2024-02-18T01:23:50.298806+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:50.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:50 smithi105 bash[27669]: audit 2024-02-18T01:23:50.371231+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.369364+0000 mon.c (mon.2) 161 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:50.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: cluster 2024-02-18T01:23:49.621751+0000 osd.7 (osd.7) 7 : cluster [DBG] 18.2 deep-scrub starts 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: cluster 2024-02-18T01:23:49.624938+0000 osd.7 (osd.7) 8 : cluster [DBG] 18.2 deep-scrub ok 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.724099+0000 mon.a (mon.0) 1059 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.724178+0000 mon.a (mon.0) 1060 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache", "force_nonempty":""}]': finished 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: cluster 2024-02-18T01:23:49.724260+0000 mon.a (mon.0) 1061 : cluster [DBG] osdmap e104: 8 total, 8 up, 8 in 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.739709+0000 mon.b (mon.1) 103 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.740091+0000 mon.b (mon.1) 104 : audit [INF] from='client.? v1:172.21.15.105:0/4134174628' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.740556+0000 mon.b (mon.1) 105 : audit [INF] from='client.? v1:172.21.15.105:0/3476391272' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.740852+0000 mon.a (mon.0) 1062 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.742305+0000 mon.a (mon.0) 1063 : audit [INF] from='client.44710 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:49.742551+0000 mon.a (mon.0) 1064 : audit [INF] from='client.43870 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: cluster 2024-02-18T01:23:50.295047+0000 osd.0 (osd.0) 11 : cluster [DBG] 18.4 deep-scrub starts 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: cluster 2024-02-18T01:23:50.298806+0000 osd.0 (osd.0) 12 : cluster [DBG] 18.4 deep-scrub ok 2024-02-18T01:23:50.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:50 smithi155 bash[20805]: audit 2024-02-18T01:23:50.371231+0000 mon.c (mon.2) 162 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.729749+0000 mon.a (mon.0) 1065 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.729823+0000 mon.a (mon.0) 1066 : audit [INF] from='client.44710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.729881+0000 mon.a (mon.0) 1067 : audit [INF] from='client.43870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: cluster 2024-02-18T01:23:50.729995+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.751622+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.753026+0000 mon.a (mon.0) 1069 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.785446+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:50.785887+0000 mon.a (mon.0) 1070 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: cluster 2024-02-18T01:23:50.941224+0000 mgr.y (mgr.24475) 278 : cluster [DBG] pgmap v234: 492 pgs: 128 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:51.373156+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[19734]: audit 2024-02-18T01:23:51.500737+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.729749+0000 mon.a (mon.0) 1065 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.729823+0000 mon.a (mon.0) 1066 : audit [INF] from='client.44710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.729881+0000 mon.a (mon.0) 1067 : audit [INF] from='client.43870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: cluster 2024-02-18T01:23:50.729995+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.751622+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.753026+0000 mon.a (mon.0) 1069 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.785446+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:50.785887+0000 mon.a (mon.0) 1070 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: cluster 2024-02-18T01:23:50.941224+0000 mgr.y (mgr.24475) 278 : cluster [DBG] pgmap v234: 492 pgs: 128 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:51.373156+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:51 smithi105 bash[27669]: audit 2024-02-18T01:23:51.500737+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.729749+0000 mon.a (mon.0) 1065 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61858-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.729823+0000 mon.a (mon.0) 1066 : audit [INF] from='client.44710 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi105-61657-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.729881+0000 mon.a (mon.0) 1067 : audit [INF] from='client.43870 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi105-61649-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: cluster 2024-02-18T01:23:50.729995+0000 mon.a (mon.0) 1068 : cluster [DBG] osdmap e105: 8 total, 8 up, 8 in 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.751622+0000 mon.b (mon.1) 106 : audit [INF] from='client.? v1:172.21.15.105:0/3933156598' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.753026+0000 mon.a (mon.0) 1069 : audit [INF] from='client.42181 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]: dispatch 2024-02-18T01:23:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.785446+0000 mon.b (mon.1) 107 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:50.785887+0000 mon.a (mon.0) 1070 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:23:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: cluster 2024-02-18T01:23:50.941224+0000 mgr.y (mgr.24475) 278 : cluster [DBG] pgmap v234: 492 pgs: 128 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 18 MiB/s wr, 5 op/s 2024-02-18T01:23:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:51.373156+0000 mon.c (mon.2) 163 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:51 smithi155 bash[20805]: audit 2024-02-18T01:23:51.500737+0000 mon.a (mon.0) 1071 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.755112+0000 mon.a (mon.0) 1072 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]': finished 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.755232+0000 mon.a (mon.0) 1073 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: cluster 2024-02-18T01:23:51.755304+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.768564+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.770128+0000 mon.a (mon.0) 1075 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.771160+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[19734]: audit 2024-02-18T01:23:51.774098+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.755112+0000 mon.a (mon.0) 1072 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]': finished 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.755232+0000 mon.a (mon.0) 1073 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: cluster 2024-02-18T01:23:51.755304+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.768564+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.770128+0000 mon.a (mon.0) 1075 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.771160+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:52 smithi105 bash[27669]: audit 2024-02-18T01:23:51.774098+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.755112+0000 mon.a (mon.0) 1072 : audit [INF] from='client.42181 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61858-10", "tierpool":"test-rados-api-smithi105-61858-10-cache"}]': finished 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.755232+0000 mon.a (mon.0) 1073 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: cluster 2024-02-18T01:23:51.755304+0000 mon.a (mon.0) 1074 : cluster [DBG] osdmap e106: 8 total, 8 up, 8 in 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.768564+0000 mon.b (mon.1) 108 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.770128+0000 mon.a (mon.0) 1075 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.771160+0000 mon.b (mon.1) 109 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:52 smithi155 bash[20805]: audit 2024-02-18T01:23:51.774098+0000 mon.a (mon.0) 1076 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:53.846 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: Running main() from gmock_main.cc 2024-02-18T01:23:53.846 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-02-18T01:23:53.846 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [----------] Global test environment set-up. 2024-02-18T01:23:53.846 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-02-18T01:23:53.846 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolList (12943 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (4157 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (4246 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (3252 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3620 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (6323 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5788 ms) 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-02-18T01:23:53.847 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (5056 ms) 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8437 ms) 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [----------] 9 tests from LibRadosPools (53822 ms total) 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [----------] Global test environment tear-down 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (53822 ms total) 2024-02-18T01:23:53.848 INFO:tasks.workunit.client.0.smithi105.stdout: api_pool: [ PASSED ] 9 tests. 2024-02-18T01:23:53.880 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-02-18T01:23:53.880 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] Global test environment set-up. 2024-02-18T01:23:53.880 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-02-18T01:23:53.880 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-02-18T01:23:53.880 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:00.057+0000 7f431689b580 -1 failed for service _ceph-mon._tcp 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:00.057+0000 7f431689b580 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (75 ms) 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2018 ms) 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2093 ms total) 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:53.881 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: started 0x56327306f530 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: started 0x563273078a10 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: started 2 aios 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: waiting 0x56327306f530 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: waiting 0x563273078a10 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: done. 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (12883 ms) 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (12883 ms total) 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-02-18T01:23:53.882 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.Exec (53 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (19 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (6 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.Applications (5068 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (27157 ms) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] 8 tests from LibRadosMisc (32303 ms total) 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [----------] Global test environment tear-down 2024-02-18T01:23:53.883 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (53865 ms total) 2024-02-18T01:23:53.884 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc: [ PASSED ] 12 tests. 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:51.648281+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:51.654560+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.376189+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.639918+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.645264+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.682815+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.686194+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.824531+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.824634+0000 mon.a (mon.0) 1078 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.824823+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T01:23:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.827317+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.829650+0000 mon.a (mon.0) 1080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.829699+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.830176+0000 mon.a (mon.0) 1081 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.835105+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.105:0/2064676468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.835653+0000 mon.c (mon.2) 166 : audit [INF] from='client.? v1:172.21.15.105:0/1356441229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.836737+0000 mon.a (mon.0) 1082 : audit [INF] from='client.46847 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:52.838475+0000 mon.a (mon.0) 1083 : audit [INF] from='client.46841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:51.648281+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:51.654560+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.376189+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.639918+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.645264+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.682815+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.686194+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.824531+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.824634+0000 mon.a (mon.0) 1078 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.824823+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.827317+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.829650+0000 mon.a (mon.0) 1080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.829699+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.830176+0000 mon.a (mon.0) 1081 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.835105+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.105:0/2064676468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.835653+0000 mon.c (mon.2) 166 : audit [INF] from='client.? v1:172.21.15.105:0/1356441229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.836737+0000 mon.a (mon.0) 1082 : audit [INF] from='client.46847 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:52.838475+0000 mon.a (mon.0) 1083 : audit [INF] from='client.46841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: cluster 2024-02-18T01:23:52.942414+0000 mgr.y (mgr.24475) 279 : cluster [DBG] pgmap v237: 452 pgs: 64 unknown, 388 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 2 op/s 2024-02-18T01:23:54.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[19734]: audit 2024-02-18T01:23:53.378839+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:54.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: cluster 2024-02-18T01:23:52.942414+0000 mgr.y (mgr.24475) 279 : cluster [DBG] pgmap v237: 452 pgs: 64 unknown, 388 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 2 op/s 2024-02-18T01:23:54.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:53 smithi105 bash[27669]: audit 2024-02-18T01:23:53.378839+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:51.648281+0000 osd.1 (osd.1) 7 : cluster [DBG] 18.3 deep-scrub starts 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:51.654560+0000 osd.1 (osd.1) 8 : cluster [DBG] 18.3 deep-scrub ok 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.376189+0000 mon.c (mon.2) 164 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.639918+0000 osd.1 (osd.1) 9 : cluster [DBG] 18.5 deep-scrub starts 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.645264+0000 osd.1 (osd.1) 10 : cluster [DBG] 18.5 deep-scrub ok 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.682815+0000 osd.7 (osd.7) 9 : cluster [DBG] 18.6 deep-scrub starts 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.686194+0000 osd.7 (osd.7) 10 : cluster [DBG] 18.6 deep-scrub ok 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.824531+0000 mon.a (mon.0) 1077 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.824634+0000 mon.a (mon.0) 1078 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.824823+0000 mon.a (mon.0) 1079 : cluster [DBG] osdmap e107: 8 total, 8 up, 8 in 2024-02-18T01:23:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.827317+0000 mon.b (mon.1) 110 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.829650+0000 mon.a (mon.0) 1080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.829699+0000 mon.b (mon.1) 111 : audit [INF] from='client.? v1:172.21.15.105:0/456018463' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.830176+0000 mon.a (mon.0) 1081 : audit [INF] from='client.34450 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.835105+0000 mon.c (mon.2) 165 : audit [INF] from='client.? v1:172.21.15.105:0/2064676468' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.835653+0000 mon.c (mon.2) 166 : audit [INF] from='client.? v1:172.21.15.105:0/1356441229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.836737+0000 mon.a (mon.0) 1082 : audit [INF] from='client.46847 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:52.838475+0000 mon.a (mon.0) 1083 : audit [INF] from='client.46841 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: cluster 2024-02-18T01:23:52.942414+0000 mgr.y (mgr.24475) 279 : cluster [DBG] pgmap v237: 452 pgs: 64 unknown, 388 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1023 B/s rd, 257 KiB/s wr, 2 op/s 2024-02-18T01:23:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:53 smithi155 bash[20805]: audit 2024-02-18T01:23:53.378839+0000 mon.c (mon.2) 167 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: cluster 2024-02-18T01:23:53.830159+0000 mon.a (mon.0) 1084 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: cluster 2024-02-18T01:23:53.830638+0000 mon.a (mon.0) 1085 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.839829+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]': finished 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.839956+0000 mon.a (mon.0) 1087 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.840054+0000 mon.a (mon.0) 1088 : audit [INF] from='client.46847 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.840162+0000 mon.a (mon.0) 1089 : audit [INF] from='client.46841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: cluster 2024-02-18T01:23:53.840333+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.912984+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.915149+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:53.915568+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: cluster 2024-02-18T01:23:54.297061+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: cluster 2024-02-18T01:23:54.299512+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[27669]: audit 2024-02-18T01:23:54.380392+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: cluster 2024-02-18T01:23:53.830159+0000 mon.a (mon.0) 1084 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:55.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: cluster 2024-02-18T01:23:53.830638+0000 mon.a (mon.0) 1085 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.839829+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]': finished 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.839956+0000 mon.a (mon.0) 1087 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.840054+0000 mon.a (mon.0) 1088 : audit [INF] from='client.46847 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.840162+0000 mon.a (mon.0) 1089 : audit [INF] from='client.46841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: cluster 2024-02-18T01:23:53.840333+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.912984+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.915149+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:53.915568+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:55.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: cluster 2024-02-18T01:23:54.297061+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:55.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: cluster 2024-02-18T01:23:54.299512+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:55.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:54 smithi105 bash[19734]: audit 2024-02-18T01:23:54.380392+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: cluster 2024-02-18T01:23:53.830159+0000 mon.a (mon.0) 1084 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:23:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: cluster 2024-02-18T01:23:53.830638+0000 mon.a (mon.0) 1085 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:23:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.839829+0000 mon.a (mon.0) 1086 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-11", "mode": "writeback"}]': finished 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.839956+0000 mon.a (mon.0) 1087 : audit [INF] from='client.34450 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi105-61891-9"}]': finished 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.840054+0000 mon.a (mon.0) 1088 : audit [INF] from='client.46847 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi105-61649-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.840162+0000 mon.a (mon.0) 1089 : audit [INF] from='client.46841 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: cluster 2024-02-18T01:23:53.840333+0000 mon.a (mon.0) 1090 : cluster [DBG] osdmap e108: 8 total, 8 up, 8 in 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.912984+0000 mon.a (mon.0) 1091 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.915149+0000 mon.a (mon.0) 1092 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:55.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:53.915568+0000 mon.a (mon.0) 1093 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:23:55.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: cluster 2024-02-18T01:23:54.297061+0000 osd.0 (osd.0) 13 : cluster [DBG] 18.8 deep-scrub starts 2024-02-18T01:23:55.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: cluster 2024-02-18T01:23:54.299512+0000 osd.0 (osd.0) 14 : cluster [DBG] 18.8 deep-scrub ok 2024-02-18T01:23:55.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:54 smithi155 bash[20805]: audit 2024-02-18T01:23:54.380392+0000 mon.c (mon.2) 168 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:56.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: audit 2024-02-18T01:23:54.868104+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:56.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: cluster 2024-02-18T01:23:54.868590+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: audit 2024-02-18T01:23:54.871752+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: audit 2024-02-18T01:23:54.928535+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: cluster 2024-02-18T01:23:54.943165+0000 mgr.y (mgr.24475) 280 : cluster [DBG] pgmap v240: 324 pgs: 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 255 KiB/s wr, 0 op/s 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: audit 2024-02-18T01:23:55.381986+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[27669]: audit 2024-02-18T01:23:55.761966+0000 mon.a (mon.0) 1097 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: audit 2024-02-18T01:23:54.868104+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: cluster 2024-02-18T01:23:54.868590+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T01:23:56.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: audit 2024-02-18T01:23:54.871752+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: audit 2024-02-18T01:23:54.928535+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: cluster 2024-02-18T01:23:54.943165+0000 mgr.y (mgr.24475) 280 : cluster [DBG] pgmap v240: 324 pgs: 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 255 KiB/s wr, 0 op/s 2024-02-18T01:23:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: audit 2024-02-18T01:23:55.381986+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[19734]: audit 2024-02-18T01:23:55.761966+0000 mon.a (mon.0) 1097 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: audit 2024-02-18T01:23:54.868104+0000 mon.a (mon.0) 1094 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:23:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: cluster 2024-02-18T01:23:54.868590+0000 mon.a (mon.0) 1095 : cluster [DBG] osdmap e109: 8 total, 8 up, 8 in 2024-02-18T01:23:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: audit 2024-02-18T01:23:54.871752+0000 mon.a (mon.0) 1096 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:23:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: audit 2024-02-18T01:23:54.928535+0000 mon.b (mon.1) 112 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: cluster 2024-02-18T01:23:54.943165+0000 mgr.y (mgr.24475) 280 : cluster [DBG] pgmap v240: 324 pgs: 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 511 B/s rd, 255 KiB/s wr, 0 op/s 2024-02-18T01:23:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: audit 2024-02-18T01:23:55.381986+0000 mon.c (mon.2) 169 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:56 smithi155 bash[20805]: audit 2024-02-18T01:23:55.761966+0000 mon.a (mon.0) 1097 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:23:56.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:23:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:23:56] "GET /metrics HTTP/1.1" 200 205371 "" "Prometheus/2.33.4" 2024-02-18T01:23:57.604 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.280941+0000 mon.a (mon.0) 1098 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:57.604 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: cluster 2024-02-18T01:23:56.281114+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T01:23:57.604 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.282257+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.283595+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3466302763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.283881+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.105:0/589363162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.285841+0000 mon.a (mon.0) 1100 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.293359+0000 mon.a (mon.0) 1101 : audit [INF] from='client.48448 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.293601+0000 mon.a (mon.0) 1102 : audit [INF] from='client.48451 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.383380+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.384532+0000 mon.c (mon.2) 171 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.385068+0000 mon.a (mon.0) 1103 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: cluster 2024-02-18T01:23:56.866047+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.875937+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.876235+0000 mon.a (mon.0) 1106 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.876591+0000 mon.a (mon.0) 1107 : audit [INF] from='client.48448 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.876709+0000 mon.a (mon.0) 1108 : audit [INF] from='client.48451 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.605 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.876837+0000 mon.a (mon.0) 1109 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: cluster 2024-02-18T01:23:56.876936+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.880607+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.882550+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.883605+0000 mon.c (mon.2) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: audit 2024-02-18T01:23:56.891951+0000 mon.a (mon.0) 1111 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.606 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[20805]: cluster 2024-02-18T01:23:56.944050+0000 mgr.y (mgr.24475) 281 : cluster [DBG] pgmap v243: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.280941+0000 mon.a (mon.0) 1098 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: cluster 2024-02-18T01:23:56.281114+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T01:23:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.282257+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.283595+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3466302763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.283881+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.105:0/589363162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.285841+0000 mon.a (mon.0) 1100 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.293359+0000 mon.a (mon.0) 1101 : audit [INF] from='client.48448 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.293601+0000 mon.a (mon.0) 1102 : audit [INF] from='client.48451 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.383380+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.384532+0000 mon.c (mon.2) 171 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.385068+0000 mon.a (mon.0) 1103 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: cluster 2024-02-18T01:23:56.866047+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.875937+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:23:57.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.876235+0000 mon.a (mon.0) 1106 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.280941+0000 mon.a (mon.0) 1098 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: cluster 2024-02-18T01:23:56.281114+0000 mon.a (mon.0) 1099 : cluster [DBG] osdmap e110: 8 total, 8 up, 8 in 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.282257+0000 mon.b (mon.1) 113 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.283595+0000 mon.b (mon.1) 114 : audit [INF] from='client.? v1:172.21.15.105:0/3466302763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.283881+0000 mon.b (mon.1) 115 : audit [INF] from='client.? v1:172.21.15.105:0/589363162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.285841+0000 mon.a (mon.0) 1100 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.293359+0000 mon.a (mon.0) 1101 : audit [INF] from='client.48448 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.293601+0000 mon.a (mon.0) 1102 : audit [INF] from='client.48451 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.383380+0000 mon.c (mon.2) 170 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.384532+0000 mon.c (mon.2) 171 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.385068+0000 mon.a (mon.0) 1103 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:23:57.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: cluster 2024-02-18T01:23:56.866047+0000 mon.a (mon.0) 1104 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.875937+0000 mon.a (mon.0) 1105 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi105-61891-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.876235+0000 mon.a (mon.0) 1106 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-11"}]': finished 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.876591+0000 mon.a (mon.0) 1107 : audit [INF] from='client.48448 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.876709+0000 mon.a (mon.0) 1108 : audit [INF] from='client.48451 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.876837+0000 mon.a (mon.0) 1109 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: cluster 2024-02-18T01:23:56.876936+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.880607+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.882550+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.883605+0000 mon.c (mon.2) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: audit 2024-02-18T01:23:56.891951+0000 mon.a (mon.0) 1111 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[19734]: cluster 2024-02-18T01:23:56.944050+0000 mgr.y (mgr.24475) 281 : cluster [DBG] pgmap v243: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.876591+0000 mon.a (mon.0) 1107 : audit [INF] from='client.48448 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi105-61657-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.876709+0000 mon.a (mon.0) 1108 : audit [INF] from='client.48451 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi105-61649-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.876837+0000 mon.a (mon.0) 1109 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: cluster 2024-02-18T01:23:56.876936+0000 mon.a (mon.0) 1110 : cluster [DBG] osdmap e111: 8 total, 8 up, 8 in 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.880607+0000 mon.c (mon.2) 172 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.882550+0000 mon.c (mon.2) 173 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.883605+0000 mon.c (mon.2) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: audit 2024-02-18T01:23:56.891951+0000 mon.a (mon.0) 1111 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:23:57.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:57 smithi105 bash[27669]: cluster 2024-02-18T01:23:56.944050+0000 mgr.y (mgr.24475) 281 : cluster [DBG] pgmap v243: 396 pgs: 72 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:23:57.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:23:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:23:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[27669]: audit 2024-02-18T01:23:57.602129+0000 mgr.y (mgr.24475) 282 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[27669]: audit 2024-02-18T01:23:57.886144+0000 mon.a (mon.0) 1112 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:23:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[27669]: cluster 2024-02-18T01:23:57.886415+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T01:23:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[27669]: audit 2024-02-18T01:23:57.891253+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[19734]: audit 2024-02-18T01:23:57.602129+0000 mgr.y (mgr.24475) 282 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:59.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[19734]: audit 2024-02-18T01:23:57.886144+0000 mon.a (mon.0) 1112 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:23:59.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[19734]: cluster 2024-02-18T01:23:57.886415+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T01:23:59.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:23:59 smithi105 bash[19734]: audit 2024-02-18T01:23:57.891253+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:23:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:59 smithi155 bash[20805]: audit 2024-02-18T01:23:57.602129+0000 mgr.y (mgr.24475) 282 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:23:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:59 smithi155 bash[20805]: audit 2024-02-18T01:23:57.886144+0000 mon.a (mon.0) 1112 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:23:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:59 smithi155 bash[20805]: cluster 2024-02-18T01:23:57.886415+0000 mon.a (mon.0) 1113 : cluster [DBG] osdmap e112: 8 total, 8 up, 8 in 2024-02-18T01:23:59.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:23:59 smithi155 bash[20805]: audit 2024-02-18T01:23:57.891253+0000 mon.c (mon.2) 175 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:58.786803+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:58.793787+0000 mon.c (mon.2) 176 : audit [INF] from='client.? v1:172.21.15.105:0/1523475605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:58.892671+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: cluster 2024-02-18T01:23:58.945072+0000 mgr.y (mgr.24475) 283 : cluster [DBG] pgmap v246: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: cluster 2024-02-18T01:23:59.203504+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.207154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.207584+0000 mon.a (mon.0) 1116 : audit [INF] from='client.47714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.792827+0000 mon.a (mon.0) 1117 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.793389+0000 mon.a (mon.0) 1118 : audit [INF] from='client.47714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: cluster 2024-02-18T01:23:59.793648+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.796923+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.105:0/1925034066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.808588+0000 mon.a (mon.0) 1120 : audit [INF] from='client.48463 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[19734]: audit 2024-02-18T01:23:59.893852+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:58.786803+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:58.793787+0000 mon.c (mon.2) 176 : audit [INF] from='client.? v1:172.21.15.105:0/1523475605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:58.892671+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: cluster 2024-02-18T01:23:58.945072+0000 mgr.y (mgr.24475) 283 : cluster [DBG] pgmap v246: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: cluster 2024-02-18T01:23:59.203504+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.207154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.207584+0000 mon.a (mon.0) 1116 : audit [INF] from='client.47714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.792827+0000 mon.a (mon.0) 1117 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.793389+0000 mon.a (mon.0) 1118 : audit [INF] from='client.47714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: cluster 2024-02-18T01:23:59.793648+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.796923+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.105:0/1925034066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.808588+0000 mon.a (mon.0) 1120 : audit [INF] from='client.48463 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:00 smithi105 bash[27669]: audit 2024-02-18T01:23:59.893852+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:58.786803+0000 mon.b (mon.1) 116 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:58.793787+0000 mon.c (mon.2) 176 : audit [INF] from='client.? v1:172.21.15.105:0/1523475605' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:58.892671+0000 mon.c (mon.2) 177 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: cluster 2024-02-18T01:23:58.945072+0000 mgr.y (mgr.24475) 283 : cluster [DBG] pgmap v246: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: cluster 2024-02-18T01:23:59.203504+0000 mon.a (mon.0) 1114 : cluster [DBG] osdmap e113: 8 total, 8 up, 8 in 2024-02-18T01:24:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.207154+0000 mon.a (mon.0) 1115 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.207584+0000 mon.a (mon.0) 1116 : audit [INF] from='client.47714 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.792827+0000 mon.a (mon.0) 1117 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.793389+0000 mon.a (mon.0) 1118 : audit [INF] from='client.47714 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi105-61649-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: cluster 2024-02-18T01:23:59.793648+0000 mon.a (mon.0) 1119 : cluster [DBG] osdmap e114: 8 total, 8 up, 8 in 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.796923+0000 mon.b (mon.1) 117 : audit [INF] from='client.? v1:172.21.15.105:0/1925034066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.808588+0000 mon.a (mon.0) 1120 : audit [INF] from='client.48463 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:00.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:00 smithi155 bash[20805]: audit 2024-02-18T01:23:59.893852+0000 mon.c (mon.2) 178 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[19734]: audit 2024-02-18T01:24:00.796634+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48463 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[19734]: cluster 2024-02-18T01:24:00.797060+0000 mon.a (mon.0) 1122 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[19734]: audit 2024-02-18T01:24:00.894603+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[19734]: cluster 2024-02-18T01:24:00.945686+0000 mgr.y (mgr.24475) 284 : cluster [DBG] pgmap v249: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[27669]: audit 2024-02-18T01:24:00.796634+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48463 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[27669]: cluster 2024-02-18T01:24:00.797060+0000 mon.a (mon.0) 1122 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T01:24:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[27669]: audit 2024-02-18T01:24:00.894603+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:02.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:01 smithi105 bash[27669]: cluster 2024-02-18T01:24:00.945686+0000 mgr.y (mgr.24475) 284 : cluster [DBG] pgmap v249: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:01 smithi155 bash[20805]: audit 2024-02-18T01:24:00.796634+0000 mon.a (mon.0) 1121 : audit [INF] from='client.48463 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi105-61657-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:01 smithi155 bash[20805]: cluster 2024-02-18T01:24:00.797060+0000 mon.a (mon.0) 1122 : cluster [DBG] osdmap e115: 8 total, 8 up, 8 in 2024-02-18T01:24:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:01 smithi155 bash[20805]: audit 2024-02-18T01:24:00.894603+0000 mon.c (mon.2) 179 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:01 smithi155 bash[20805]: cluster 2024-02-18T01:24:00.945686+0000 mgr.y (mgr.24475) 284 : cluster [DBG] pgmap v249: 364 pgs: 64 unknown, 8 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 3 op/s 2024-02-18T01:24:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[19734]: cluster 2024-02-18T01:24:01.802557+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T01:24:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[19734]: audit 2024-02-18T01:24:01.808005+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[19734]: audit 2024-02-18T01:24:01.895510+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[27669]: cluster 2024-02-18T01:24:01.802557+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T01:24:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[27669]: audit 2024-02-18T01:24:01.808005+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:02 smithi105 bash[27669]: audit 2024-02-18T01:24:01.895510+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:02 smithi155 bash[20805]: cluster 2024-02-18T01:24:01.802557+0000 mon.a (mon.0) 1123 : cluster [DBG] osdmap e116: 8 total, 8 up, 8 in 2024-02-18T01:24:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:02 smithi155 bash[20805]: audit 2024-02-18T01:24:01.808005+0000 mon.a (mon.0) 1124 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:02 smithi155 bash[20805]: audit 2024-02-18T01:24:01.895510+0000 mon.c (mon.2) 180 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: audit 2024-02-18T01:24:02.809266+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: cluster 2024-02-18T01:24:02.809851+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: audit 2024-02-18T01:24:02.812135+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.105:0/189704025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: audit 2024-02-18T01:24:02.821835+0000 mon.a (mon.0) 1127 : audit [INF] from='client.48472 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: audit 2024-02-18T01:24:02.896426+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: cluster 2024-02-18T01:24:02.946303+0000 mgr.y (mgr.24475) 285 : cluster [DBG] pgmap v252: 396 pgs: 32 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:04.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: audit 2024-02-18T01:24:03.787449+0000 mon.a (mon.0) 1128 : audit [INF] from='client.48472 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[19734]: cluster 2024-02-18T01:24:03.788656+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: audit 2024-02-18T01:24:02.809266+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: cluster 2024-02-18T01:24:02.809851+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: audit 2024-02-18T01:24:02.812135+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.105:0/189704025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: audit 2024-02-18T01:24:02.821835+0000 mon.a (mon.0) 1127 : audit [INF] from='client.48472 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: audit 2024-02-18T01:24:02.896426+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: cluster 2024-02-18T01:24:02.946303+0000 mgr.y (mgr.24475) 285 : cluster [DBG] pgmap v252: 396 pgs: 32 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: audit 2024-02-18T01:24:03.787449+0000 mon.a (mon.0) 1128 : audit [INF] from='client.48472 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:04 smithi105 bash[27669]: cluster 2024-02-18T01:24:03.788656+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: audit 2024-02-18T01:24:02.809266+0000 mon.a (mon.0) 1125 : audit [INF] from='client.? v1:172.21.15.105:0/445649619' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi105-61649-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: cluster 2024-02-18T01:24:02.809851+0000 mon.a (mon.0) 1126 : cluster [DBG] osdmap e117: 8 total, 8 up, 8 in 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: audit 2024-02-18T01:24:02.812135+0000 mon.b (mon.1) 118 : audit [INF] from='client.? v1:172.21.15.105:0/189704025' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: audit 2024-02-18T01:24:02.821835+0000 mon.a (mon.0) 1127 : audit [INF] from='client.48472 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: audit 2024-02-18T01:24:02.896426+0000 mon.c (mon.2) 181 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: cluster 2024-02-18T01:24:02.946303+0000 mgr.y (mgr.24475) 285 : cluster [DBG] pgmap v252: 396 pgs: 32 unknown, 32 creating+peering, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: audit 2024-02-18T01:24:03.787449+0000 mon.a (mon.0) 1128 : audit [INF] from='client.48472 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi105-61657-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:04.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:04 smithi155 bash[20805]: cluster 2024-02-18T01:24:03.788656+0000 mon.a (mon.0) 1129 : cluster [DBG] osdmap e118: 8 total, 8 up, 8 in 2024-02-18T01:24:05.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: cluster 2024-02-18T01:24:03.825322+0000 mon.a (mon.0) 1130 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:05.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:03.897228+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:03.977649+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.271001+0000 mon.a (mon.0) 1131 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.789699+0000 mon.a (mon.0) 1132 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: cluster 2024-02-18T01:24:04.789941+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.791286+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.791799+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.105:0/1928944696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.792865+0000 mon.a (mon.0) 1134 : audit [INF] from='client.47732 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.799445+0000 mon.a (mon.0) 1135 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: audit 2024-02-18T01:24:04.898309+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[19734]: cluster 2024-02-18T01:24:04.947117+0000 mgr.y (mgr.24475) 286 : cluster [DBG] pgmap v255: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: cluster 2024-02-18T01:24:03.825322+0000 mon.a (mon.0) 1130 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:03.897228+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:03.977649+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.271001+0000 mon.a (mon.0) 1131 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.789699+0000 mon.a (mon.0) 1132 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: cluster 2024-02-18T01:24:04.789941+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.791286+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.791799+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.105:0/1928944696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.792865+0000 mon.a (mon.0) 1134 : audit [INF] from='client.47732 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.799445+0000 mon.a (mon.0) 1135 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: audit 2024-02-18T01:24:04.898309+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:05 smithi105 bash[27669]: cluster 2024-02-18T01:24:04.947117+0000 mgr.y (mgr.24475) 286 : cluster [DBG] pgmap v255: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: cluster 2024-02-18T01:24:03.825322+0000 mon.a (mon.0) 1130 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:03.897228+0000 mon.c (mon.2) 182 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:03.977649+0000 mon.b (mon.1) 119 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.271001+0000 mon.a (mon.0) 1131 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.789699+0000 mon.a (mon.0) 1132 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: cluster 2024-02-18T01:24:04.789941+0000 mon.a (mon.0) 1133 : cluster [DBG] osdmap e119: 8 total, 8 up, 8 in 2024-02-18T01:24:05.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.791286+0000 mon.b (mon.1) 120 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.791799+0000 mon.c (mon.2) 183 : audit [INF] from='client.? v1:172.21.15.105:0/1928944696' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.792865+0000 mon.a (mon.0) 1134 : audit [INF] from='client.47732 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.799445+0000 mon.a (mon.0) 1135 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: audit 2024-02-18T01:24:04.898309+0000 mon.c (mon.2) 184 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:05.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:05 smithi155 bash[20805]: cluster 2024-02-18T01:24:04.947117+0000 mgr.y (mgr.24475) 286 : cluster [DBG] pgmap v255: 364 pgs: 32 unknown, 332 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 81 KiB/s wr, 77 op/s 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.799447+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47732 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.799573+0000 mon.a (mon.0) 1137 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: cluster 2024-02-18T01:24:05.799818+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.807624+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.811345+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.812740+0000 mon.a (mon.0) 1140 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.817061+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[19734]: audit 2024-02-18T01:24:05.899848+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.799447+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47732 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.799573+0000 mon.a (mon.0) 1137 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: cluster 2024-02-18T01:24:05.799818+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.807624+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.811345+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.812740+0000 mon.a (mon.0) 1140 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.817061+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[27669]: audit 2024-02-18T01:24:05.899848+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:06.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:06] "GET /metrics HTTP/1.1" 200 196395 "" "Prometheus/2.33.4" 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.799447+0000 mon.a (mon.0) 1136 : audit [INF] from='client.47732 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi105-61649-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.799573+0000 mon.a (mon.0) 1137 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: cluster 2024-02-18T01:24:05.799818+0000 mon.a (mon.0) 1138 : cluster [DBG] osdmap e120: 8 total, 8 up, 8 in 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.807624+0000 mon.b (mon.1) 121 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.811345+0000 mon.a (mon.0) 1139 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.812740+0000 mon.a (mon.0) 1140 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]: dispatch 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.817061+0000 mon.a (mon.0) 1141 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:06 smithi155 bash[20805]: audit 2024-02-18T01:24:05.899848+0000 mon.c (mon.2) 185 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:07.821 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: Running main() from gmock_main.cc 2024-02-18T01:24:07.821 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] Global test environment set-up. 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (3507 ms) 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (4864 ms) 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2008 ms) 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1374 ms) 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (11753 ms total) 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: 2024-02-18T01:24:07.822 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (4944 ms) 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (4328 ms) 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (9272 ms total) 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3080 ms) 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (2005 ms) 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-02-18T01:24:07.823 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2159 ms) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (3173 ms) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (10417 ms total) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (3955 ms) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (3983 ms) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (7938 ms total) 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: 2024-02-18T01:24:07.824 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [----------] Global test environment tear-down 2024-02-18T01:24:07.825 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (67779 ms total) 2024-02-18T01:24:07.825 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62133 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62133 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61676 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61676 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61970 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61970 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61910 2024-02-18T01:24:07.826 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61910 2024-02-18T01:24:07.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:07.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: cluster 2024-02-18T01:24:06.794195+0000 mon.a (mon.0) 1142 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:07.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:06.805821+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:06.806040+0000 mon.a (mon.0) 1144 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]': finished 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:06.806252+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: cluster 2024-02-18T01:24:06.806578+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:06.817704+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:06.901252+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: cluster 2024-02-18T01:24:06.947708+0000 mgr.y (mgr.24475) 287 : cluster [DBG] pgmap v258: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.627203+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.627921+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.628477+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.629061+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.629705+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.630380+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:07.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.630953+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.631528+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.632088+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.632699+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.810613+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: cluster 2024-02-18T01:24:07.811098+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.813105+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.105:0/2239002935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:07.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:07 smithi155 bash[20805]: audit 2024-02-18T01:24:07.817051+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: cluster 2024-02-18T01:24:06.794195+0000 mon.a (mon.0) 1142 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:06.805821+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:06.806040+0000 mon.a (mon.0) 1144 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]': finished 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:06.806252+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: cluster 2024-02-18T01:24:06.806578+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:06.817704+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:06.901252+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: cluster 2024-02-18T01:24:06.947708+0000 mgr.y (mgr.24475) 287 : cluster [DBG] pgmap v258: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.627203+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.627921+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.628477+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.629061+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.629705+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.630380+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.630953+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:08.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.631528+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.632088+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: cluster 2024-02-18T01:24:06.794195+0000 mon.a (mon.0) 1142 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:06.805821+0000 mon.a (mon.0) 1143 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:06.806040+0000 mon.a (mon.0) 1144 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-13", "mode": "writeback"}]': finished 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:06.806252+0000 mon.a (mon.0) 1145 : audit [INF] from='client.? v1:172.21.15.105:0/237714174' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi105-61657-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: cluster 2024-02-18T01:24:06.806578+0000 mon.a (mon.0) 1146 : cluster [DBG] osdmap e121: 8 total, 8 up, 8 in 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:06.817704+0000 mon.a (mon.0) 1147 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]: dispatch 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:06.901252+0000 mon.c (mon.2) 186 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: cluster 2024-02-18T01:24:06.947708+0000 mgr.y (mgr.24475) 287 : cluster [DBG] pgmap v258: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:08.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.627203+0000 mon.b (mon.1) 122 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.627921+0000 mon.b (mon.1) 123 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.628477+0000 mon.b (mon.1) 124 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.629061+0000 mon.b (mon.1) 125 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.629705+0000 mon.b (mon.1) 126 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.630380+0000 mon.b (mon.1) 127 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.630953+0000 mon.b (mon.1) 128 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.631528+0000 mon.b (mon.1) 129 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.632088+0000 mon.b (mon.1) 130 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.632699+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:08.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.810613+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: cluster 2024-02-18T01:24:07.811098+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.813105+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.105:0/2239002935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[19734]: audit 2024-02-18T01:24:07.817051+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.632699+0000 mon.b (mon.1) 131 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.810613+0000 mon.a (mon.0) 1148 : audit [INF] from='client.? v1:172.21.15.105:0/4178959505' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi105-61891-14"}]': finished 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: cluster 2024-02-18T01:24:07.811098+0000 mon.a (mon.0) 1149 : cluster [DBG] osdmap e122: 8 total, 8 up, 8 in 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.813105+0000 mon.b (mon.1) 132 : audit [INF] from='client.? v1:172.21.15.105:0/2239002935' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:08.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:07 smithi105 bash[27669]: audit 2024-02-18T01:24:07.817051+0000 mon.a (mon.0) 1150 : audit [INF] from='client.48484 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.613033+0000 mgr.y (mgr.24475) 288 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.627418+0000 mgr.y (mgr.24475) 289 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.628065+0000 mgr.y (mgr.24475) 290 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.628626+0000 mgr.y (mgr.24475) 291 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.629224+0000 mgr.y (mgr.24475) 292 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.629878+0000 mgr.y (mgr.24475) 293 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.630551+0000 mgr.y (mgr.24475) 294 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.631100+0000 mgr.y (mgr.24475) 295 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.631687+0000 mgr.y (mgr.24475) 296 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.632243+0000 mgr.y (mgr.24475) 297 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.632861+0000 mgr.y (mgr.24475) 298 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:07.902371+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:08.795138+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.795577+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:08.800294+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.105:0/1459807242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[27669]: audit 2024-02-18T01:24:08.802119+0000 mon.a (mon.0) 1153 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.613033+0000 mgr.y (mgr.24475) 288 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.627418+0000 mgr.y (mgr.24475) 289 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.628065+0000 mgr.y (mgr.24475) 290 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.628626+0000 mgr.y (mgr.24475) 291 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.629224+0000 mgr.y (mgr.24475) 292 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.629878+0000 mgr.y (mgr.24475) 293 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.630551+0000 mgr.y (mgr.24475) 294 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.631100+0000 mgr.y (mgr.24475) 295 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.631687+0000 mgr.y (mgr.24475) 296 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.632243+0000 mgr.y (mgr.24475) 297 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.632861+0000 mgr.y (mgr.24475) 298 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:07.902371+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:09.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:08.795138+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:09.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.795577+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T01:24:09.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:08.800294+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.105:0/1459807242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:08 smithi105 bash[19734]: audit 2024-02-18T01:24:08.802119+0000 mon.a (mon.0) 1153 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.613033+0000 mgr.y (mgr.24475) 288 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.627418+0000 mgr.y (mgr.24475) 289 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.0"}]: dispatch 2024-02-18T01:24:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.628065+0000 mgr.y (mgr.24475) 290 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.1"}]: dispatch 2024-02-18T01:24:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.628626+0000 mgr.y (mgr.24475) 291 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.2"}]: dispatch 2024-02-18T01:24:09.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.629224+0000 mgr.y (mgr.24475) 292 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.3"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.629878+0000 mgr.y (mgr.24475) 293 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.4"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.630551+0000 mgr.y (mgr.24475) 294 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.5"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.631100+0000 mgr.y (mgr.24475) 295 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.6"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.631687+0000 mgr.y (mgr.24475) 296 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.7"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.632243+0000 mgr.y (mgr.24475) 297 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.8"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.632861+0000 mgr.y (mgr.24475) 298 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "95.9"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:07.902371+0000 mon.c (mon.2) 187 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:08.795138+0000 mon.a (mon.0) 1151 : audit [INF] from='client.48484 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi105-61649-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.795577+0000 mon.a (mon.0) 1152 : cluster [DBG] osdmap e123: 8 total, 8 up, 8 in 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:08.800294+0000 mon.b (mon.1) 133 : audit [INF] from='client.? v1:172.21.15.105:0/1459807242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:09.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:08 smithi155 bash[20805]: audit 2024-02-18T01:24:08.802119+0000 mon.a (mon.0) 1153 : audit [INF] from='client.48487 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.612696+0000 osd.1 (osd.1) 11 : cluster [DBG] 95.8 scrub starts 2024-02-18T01:24:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.616853+0000 osd.1 (osd.1) 12 : cluster [DBG] 95.8 scrub ok 2024-02-18T01:24:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.624365+0000 osd.4 (osd.4) 3 : cluster [DBG] 95.1 scrub starts 2024-02-18T01:24:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.628225+0000 osd.4 (osd.4) 4 : cluster [DBG] 95.1 scrub ok 2024-02-18T01:24:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: audit 2024-02-18T01:24:08.903741+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:08.948839+0000 mgr.y (mgr.24475) 299 : cluster [DBG] pgmap v261: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 44 KiB/s wr, 101 op/s 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: audit 2024-02-18T01:24:09.792551+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: cluster 2024-02-18T01:24:09.792955+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[19734]: audit 2024-02-18T01:24:09.905062+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.612696+0000 osd.1 (osd.1) 11 : cluster [DBG] 95.8 scrub starts 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.616853+0000 osd.1 (osd.1) 12 : cluster [DBG] 95.8 scrub ok 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.624365+0000 osd.4 (osd.4) 3 : cluster [DBG] 95.1 scrub starts 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.628225+0000 osd.4 (osd.4) 4 : cluster [DBG] 95.1 scrub ok 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: audit 2024-02-18T01:24:08.903741+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:08.948839+0000 mgr.y (mgr.24475) 299 : cluster [DBG] pgmap v261: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 44 KiB/s wr, 101 op/s 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: audit 2024-02-18T01:24:09.792551+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:10.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: cluster 2024-02-18T01:24:09.792955+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T01:24:10.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:10 smithi105 bash[27669]: audit 2024-02-18T01:24:09.905062+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.612696+0000 osd.1 (osd.1) 11 : cluster [DBG] 95.8 scrub starts 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.616853+0000 osd.1 (osd.1) 12 : cluster [DBG] 95.8 scrub ok 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.624365+0000 osd.4 (osd.4) 3 : cluster [DBG] 95.1 scrub starts 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.628225+0000 osd.4 (osd.4) 4 : cluster [DBG] 95.1 scrub ok 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: audit 2024-02-18T01:24:08.903741+0000 mon.c (mon.2) 188 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:08.948839+0000 mgr.y (mgr.24475) 299 : cluster [DBG] pgmap v261: 388 pgs: 64 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 35 KiB/s rd, 44 KiB/s wr, 101 op/s 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: audit 2024-02-18T01:24:09.792551+0000 mon.a (mon.0) 1154 : audit [INF] from='client.48487 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi105-61657-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: cluster 2024-02-18T01:24:09.792955+0000 mon.a (mon.0) 1155 : cluster [DBG] osdmap e124: 8 total, 8 up, 8 in 2024-02-18T01:24:10.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:10 smithi155 bash[20805]: audit 2024-02-18T01:24:09.905062+0000 mon.c (mon.2) 189 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:09.478827+0000 osd.5 (osd.5) 3 : cluster [DBG] 95.3 scrub starts 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:09.482061+0000 osd.5 (osd.5) 4 : cluster [DBG] 95.3 scrub ok 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:09.904238+0000 osd.3 (osd.3) 11 : cluster [DBG] 95.5 scrub starts 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:09.908208+0000 osd.3 (osd.3) 12 : cluster [DBG] 95.5 scrub ok 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.288803+0000 osd.0 (osd.0) 15 : cluster [DBG] 95.4 scrub starts 2024-02-18T01:24:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.291651+0000 osd.0 (osd.0) 16 : cluster [DBG] 95.4 scrub ok 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.797714+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: audit 2024-02-18T01:24:10.805615+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.105:0/4269699307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: audit 2024-02-18T01:24:10.808886+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: audit 2024-02-18T01:24:10.906798+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.949896+0000 mgr.y (mgr.24475) 300 : cluster [DBG] pgmap v264: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 37 KiB/s rd, 50 KiB/s wr, 102 op/s 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[27669]: audit 2024-02-18T01:24:11.251656+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:09.478827+0000 osd.5 (osd.5) 3 : cluster [DBG] 95.3 scrub starts 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:09.482061+0000 osd.5 (osd.5) 4 : cluster [DBG] 95.3 scrub ok 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:09.904238+0000 osd.3 (osd.3) 11 : cluster [DBG] 95.5 scrub starts 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:09.908208+0000 osd.3 (osd.3) 12 : cluster [DBG] 95.5 scrub ok 2024-02-18T01:24:11.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.288803+0000 osd.0 (osd.0) 15 : cluster [DBG] 95.4 scrub starts 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.291651+0000 osd.0 (osd.0) 16 : cluster [DBG] 95.4 scrub ok 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.797714+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: audit 2024-02-18T01:24:10.805615+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.105:0/4269699307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: audit 2024-02-18T01:24:10.808886+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: audit 2024-02-18T01:24:10.906798+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.949896+0000 mgr.y (mgr.24475) 300 : cluster [DBG] pgmap v264: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 37 KiB/s rd, 50 KiB/s wr, 102 op/s 2024-02-18T01:24:11.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:11 smithi105 bash[19734]: audit 2024-02-18T01:24:11.251656+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:09.478827+0000 osd.5 (osd.5) 3 : cluster [DBG] 95.3 scrub starts 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:09.482061+0000 osd.5 (osd.5) 4 : cluster [DBG] 95.3 scrub ok 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:09.904238+0000 osd.3 (osd.3) 11 : cluster [DBG] 95.5 scrub starts 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:09.908208+0000 osd.3 (osd.3) 12 : cluster [DBG] 95.5 scrub ok 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.288803+0000 osd.0 (osd.0) 15 : cluster [DBG] 95.4 scrub starts 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.291651+0000 osd.0 (osd.0) 16 : cluster [DBG] 95.4 scrub ok 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.797714+0000 mon.a (mon.0) 1156 : cluster [DBG] osdmap e125: 8 total, 8 up, 8 in 2024-02-18T01:24:11.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: audit 2024-02-18T01:24:10.805615+0000 mon.c (mon.2) 190 : audit [INF] from='client.? v1:172.21.15.105:0/4269699307' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: audit 2024-02-18T01:24:10.808886+0000 mon.a (mon.0) 1157 : audit [INF] from='client.47747 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:11.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: audit 2024-02-18T01:24:10.906798+0000 mon.c (mon.2) 191 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:11.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.949896+0000 mgr.y (mgr.24475) 300 : cluster [DBG] pgmap v264: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 37 KiB/s rd, 50 KiB/s wr, 102 op/s 2024-02-18T01:24:11.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[20805]: audit 2024-02-18T01:24:11.251656+0000 mon.a (mon.0) 1158 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:24:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.846796+0000 osd.6 (osd.6) 3 : cluster [DBG] 95.0 scrub starts 2024-02-18T01:24:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: cluster 2024-02-18T01:24:10.848579+0000 osd.6 (osd.6) 4 : cluster [DBG] 95.0 scrub ok 2024-02-18T01:24:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.631300+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:24:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.632878+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: cluster 2024-02-18T01:24:11.635425+0000 mgr.y (mgr.24475) 301 : cluster [DBG] pgmap v265: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.642225+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: cluster 2024-02-18T01:24:11.790299+0000 mon.a (mon.0) 1162 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 17 stray daemon(s) not managed by cephadm) 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.802829+0000 mon.a (mon.0) 1163 : audit [INF] from='client.47747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: cluster 2024-02-18T01:24:11.802970+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.821592+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[19734]: audit 2024-02-18T01:24:11.908299+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.846796+0000 osd.6 (osd.6) 3 : cluster [DBG] 95.0 scrub starts 2024-02-18T01:24:12.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: cluster 2024-02-18T01:24:10.848579+0000 osd.6 (osd.6) 4 : cluster [DBG] 95.0 scrub ok 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.631300+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.632878+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: cluster 2024-02-18T01:24:11.635425+0000 mgr.y (mgr.24475) 301 : cluster [DBG] pgmap v265: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.642225+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: cluster 2024-02-18T01:24:11.790299+0000 mon.a (mon.0) 1162 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 17 stray daemon(s) not managed by cephadm) 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.802829+0000 mon.a (mon.0) 1163 : audit [INF] from='client.47747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: cluster 2024-02-18T01:24:11.802970+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.821592+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:12.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:12 smithi105 bash[27669]: audit 2024-02-18T01:24:11.908299+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.846796+0000 osd.6 (osd.6) 3 : cluster [DBG] 95.0 scrub starts 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: cluster 2024-02-18T01:24:10.848579+0000 osd.6 (osd.6) 4 : cluster [DBG] 95.0 scrub ok 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.631300+0000 mon.a (mon.0) 1159 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.632878+0000 mon.a (mon.0) 1160 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: cluster 2024-02-18T01:24:11.635425+0000 mgr.y (mgr.24475) 301 : cluster [DBG] pgmap v265: 356 pgs: 32 unknown, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.642225+0000 mon.a (mon.0) 1161 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: cluster 2024-02-18T01:24:11.790299+0000 mon.a (mon.0) 1162 : cluster [INF] Health check cleared: CEPHADM_STRAY_DAEMON (was: 17 stray daemon(s) not managed by cephadm) 2024-02-18T01:24:12.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.802829+0000 mon.a (mon.0) 1163 : audit [INF] from='client.47747 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi105-61649-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:12.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: cluster 2024-02-18T01:24:11.802970+0000 mon.a (mon.0) 1164 : cluster [DBG] osdmap e126: 8 total, 8 up, 8 in 2024-02-18T01:24:12.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.821592+0000 mon.a (mon.0) 1165 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:12.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:12 smithi155 bash[20805]: audit 2024-02-18T01:24:11.908299+0000 mon.c (mon.2) 192 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[27669]: cluster 2024-02-18T01:24:12.286169+0000 osd.0 (osd.0) 17 : cluster [DBG] 95.9 scrub starts 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[27669]: cluster 2024-02-18T01:24:12.291166+0000 osd.0 (osd.0) 18 : cluster [DBG] 95.9 scrub ok 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[27669]: audit 2024-02-18T01:24:12.809667+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[27669]: cluster 2024-02-18T01:24:12.809893+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[19734]: cluster 2024-02-18T01:24:12.286169+0000 osd.0 (osd.0) 17 : cluster [DBG] 95.9 scrub starts 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[19734]: cluster 2024-02-18T01:24:12.291166+0000 osd.0 (osd.0) 18 : cluster [DBG] 95.9 scrub ok 2024-02-18T01:24:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[19734]: audit 2024-02-18T01:24:12.809667+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:13 smithi105 bash[19734]: cluster 2024-02-18T01:24:12.809893+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T01:24:13.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:13 smithi155 bash[20805]: cluster 2024-02-18T01:24:12.286169+0000 osd.0 (osd.0) 17 : cluster [DBG] 95.9 scrub starts 2024-02-18T01:24:13.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:13 smithi155 bash[20805]: cluster 2024-02-18T01:24:12.291166+0000 osd.0 (osd.0) 18 : cluster [DBG] 95.9 scrub ok 2024-02-18T01:24:13.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:13 smithi155 bash[20805]: audit 2024-02-18T01:24:12.809667+0000 mon.a (mon.0) 1166 : audit [INF] from='client.? v1:172.21.15.105:0/1349324078' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi105-61657-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:13.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:13 smithi155 bash[20805]: cluster 2024-02-18T01:24:12.809893+0000 mon.a (mon.0) 1167 : cluster [DBG] osdmap e127: 8 total, 8 up, 8 in 2024-02-18T01:24:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:12.781474+0000 osd.6 (osd.6) 5 : cluster [DBG] 95.6 scrub starts 2024-02-18T01:24:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:12.786538+0000 osd.6 (osd.6) 6 : cluster [DBG] 95.6 scrub ok 2024-02-18T01:24:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: audit 2024-02-18T01:24:12.909798+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:13.636147+0000 mgr.y (mgr.24475) 302 : cluster [DBG] pgmap v268: 324 pgs: 32 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 23 op/s 2024-02-18T01:24:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:13.783416+0000 mon.a (mon.0) 1168 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:13.815461+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: audit 2024-02-18T01:24:13.821683+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: audit 2024-02-18T01:24:13.828163+0000 mon.c (mon.2) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2685416470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: audit 2024-02-18T01:24:13.833729+0000 mon.a (mon.0) 1171 : audit [INF] from='client.47765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: audit 2024-02-18T01:24:13.911034+0000 mon.c (mon.2) 195 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:14.541519+0000 osd.7 (osd.7) 11 : cluster [DBG] 95.2 scrub starts 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[19734]: cluster 2024-02-18T01:24:14.545880+0000 osd.7 (osd.7) 12 : cluster [DBG] 95.2 scrub ok 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:12.781474+0000 osd.6 (osd.6) 5 : cluster [DBG] 95.6 scrub starts 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:12.786538+0000 osd.6 (osd.6) 6 : cluster [DBG] 95.6 scrub ok 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: audit 2024-02-18T01:24:12.909798+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:13.636147+0000 mgr.y (mgr.24475) 302 : cluster [DBG] pgmap v268: 324 pgs: 32 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 23 op/s 2024-02-18T01:24:14.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:13.783416+0000 mon.a (mon.0) 1168 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:13.815461+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: audit 2024-02-18T01:24:13.821683+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: audit 2024-02-18T01:24:13.828163+0000 mon.c (mon.2) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2685416470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: audit 2024-02-18T01:24:13.833729+0000 mon.a (mon.0) 1171 : audit [INF] from='client.47765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: audit 2024-02-18T01:24:13.911034+0000 mon.c (mon.2) 195 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:14.541519+0000 osd.7 (osd.7) 11 : cluster [DBG] 95.2 scrub starts 2024-02-18T01:24:14.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:14 smithi105 bash[27669]: cluster 2024-02-18T01:24:14.545880+0000 osd.7 (osd.7) 12 : cluster [DBG] 95.2 scrub ok 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:12.781474+0000 osd.6 (osd.6) 5 : cluster [DBG] 95.6 scrub starts 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:12.786538+0000 osd.6 (osd.6) 6 : cluster [DBG] 95.6 scrub ok 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: audit 2024-02-18T01:24:12.909798+0000 mon.c (mon.2) 193 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:13.636147+0000 mgr.y (mgr.24475) 302 : cluster [DBG] pgmap v268: 324 pgs: 32 creating+peering, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 20 KiB/s rd, 23 op/s 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:13.783416+0000 mon.a (mon.0) 1168 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:13.815461+0000 mon.a (mon.0) 1169 : cluster [DBG] osdmap e128: 8 total, 8 up, 8 in 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: audit 2024-02-18T01:24:13.821683+0000 mon.a (mon.0) 1170 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: audit 2024-02-18T01:24:13.828163+0000 mon.c (mon.2) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2685416470' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: audit 2024-02-18T01:24:13.833729+0000 mon.a (mon.0) 1171 : audit [INF] from='client.47765 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:14.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: audit 2024-02-18T01:24:13.911034+0000 mon.c (mon.2) 195 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:14.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:14.541519+0000 osd.7 (osd.7) 11 : cluster [DBG] 95.2 scrub starts 2024-02-18T01:24:14.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:14 smithi155 bash[20805]: cluster 2024-02-18T01:24:14.545880+0000 osd.7 (osd.7) 12 : cluster [DBG] 95.2 scrub ok 2024-02-18T01:24:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.819542+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.820215+0000 mon.a (mon.0) 1173 : audit [INF] from='client.47765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: cluster 2024-02-18T01:24:14.820507+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.823615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.105:0/314288765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.824471+0000 mon.a (mon.0) 1175 : audit [INF] from='client.47768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.869570+0000 mon.c (mon.2) 197 : audit [DBG] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.870169+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.870460+0000 mon.a (mon.0) 1176 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: audit 2024-02-18T01:24:14.912287+0000 mon.c (mon.2) 199 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: cluster 2024-02-18T01:24:15.567209+0000 osd.7 (osd.7) 13 : cluster [DBG] 95.7 scrub starts 2024-02-18T01:24:16.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[19734]: cluster 2024-02-18T01:24:15.573782+0000 osd.7 (osd.7) 14 : cluster [DBG] 95.7 scrub ok 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.819542+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.820215+0000 mon.a (mon.0) 1173 : audit [INF] from='client.47765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: cluster 2024-02-18T01:24:14.820507+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.823615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.105:0/314288765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.824471+0000 mon.a (mon.0) 1175 : audit [INF] from='client.47768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.869570+0000 mon.c (mon.2) 197 : audit [DBG] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.870169+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.870460+0000 mon.a (mon.0) 1176 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: audit 2024-02-18T01:24:14.912287+0000 mon.c (mon.2) 199 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: cluster 2024-02-18T01:24:15.567209+0000 osd.7 (osd.7) 13 : cluster [DBG] 95.7 scrub starts 2024-02-18T01:24:16.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:15 smithi105 bash[27669]: cluster 2024-02-18T01:24:15.573782+0000 osd.7 (osd.7) 14 : cluster [DBG] 95.7 scrub ok 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.819542+0000 mon.a (mon.0) 1172 : audit [INF] from='client.? v1:172.21.15.105:0/464002854' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61914-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.820215+0000 mon.a (mon.0) 1173 : audit [INF] from='client.47765 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61649-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: cluster 2024-02-18T01:24:14.820507+0000 mon.a (mon.0) 1174 : cluster [DBG] osdmap e129: 8 total, 8 up, 8 in 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.823615+0000 mon.c (mon.2) 196 : audit [INF] from='client.? v1:172.21.15.105:0/314288765' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.824471+0000 mon.a (mon.0) 1175 : audit [INF] from='client.47768 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.869570+0000 mon.c (mon.2) 197 : audit [DBG] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.870169+0000 mon.c (mon.2) 198 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.870460+0000 mon.a (mon.0) 1176 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: audit 2024-02-18T01:24:14.912287+0000 mon.c (mon.2) 199 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:16.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: cluster 2024-02-18T01:24:15.567209+0000 osd.7 (osd.7) 13 : cluster [DBG] 95.7 scrub starts 2024-02-18T01:24:16.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:15 smithi155 bash[20805]: cluster 2024-02-18T01:24:15.573782+0000 osd.7 (osd.7) 14 : cluster [DBG] 95.7 scrub ok 2024-02-18T01:24:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:16] "GET /metrics HTTP/1.1" 200 196395 "" "Prometheus/2.33.4" 2024-02-18T01:24:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: cluster 2024-02-18T01:24:15.637026+0000 mgr.y (mgr.24475) 303 : cluster [DBG] pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 22 op/s 2024-02-18T01:24:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.822893+0000 mon.a (mon.0) 1177 : audit [INF] from='client.47768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.823391+0000 mon.a (mon.0) 1178 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: cluster 2024-02-18T01:24:15.823592+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.827138+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2"}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.828942+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.847827+0000 mon.a (mon.0) 1180 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[19734]: audit 2024-02-18T01:24:15.913636+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: cluster 2024-02-18T01:24:15.637026+0000 mgr.y (mgr.24475) 303 : cluster [DBG] pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 22 op/s 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.822893+0000 mon.a (mon.0) 1177 : audit [INF] from='client.47768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.823391+0000 mon.a (mon.0) 1178 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: cluster 2024-02-18T01:24:15.823592+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.827138+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2"}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.828942+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.847827+0000 mon.a (mon.0) 1180 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:16.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:16 smithi105 bash[27669]: audit 2024-02-18T01:24:15.913636+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: cluster 2024-02-18T01:24:15.637026+0000 mgr.y (mgr.24475) 303 : cluster [DBG] pgmap v271: 388 pgs: 96 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 19 KiB/s rd, 22 op/s 2024-02-18T01:24:17.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.822893+0000 mon.a (mon.0) 1177 : audit [INF] from='client.47768 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi105-61657-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.823391+0000 mon.a (mon.0) 1178 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: cluster 2024-02-18T01:24:15.823592+0000 mon.a (mon.0) 1179 : cluster [DBG] osdmap e130: 8 total, 8 up, 8 in 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.827138+0000 mon.c (mon.2) 200 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2"}]: dispatch 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.828942+0000 mon.c (mon.2) 201 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.847827+0000 mon.a (mon.0) 1180 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:17.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:16 smithi155 bash[20805]: audit 2024-02-18T01:24:15.913636+0000 mon.c (mon.2) 202 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:17.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:17.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.858070+0000 mon.a (mon.0) 1181 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: cluster 2024-02-18T01:24:16.858315+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.859481+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.860211+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.861181+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.873918+0000 mon.a (mon.0) 1184 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:17 smithi155 bash[20805]: audit 2024-02-18T01:24:16.914617+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.858070+0000 mon.a (mon.0) 1181 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: cluster 2024-02-18T01:24:16.858315+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.859481+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.860211+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.861181+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.873918+0000 mon.a (mon.0) 1184 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[27669]: audit 2024-02-18T01:24:16.914617+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.858070+0000 mon.a (mon.0) 1181 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi105-61790-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: cluster 2024-02-18T01:24:16.858315+0000 mon.a (mon.0) 1182 : cluster [DBG] osdmap e131: 8 total, 8 up, 8 in 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.859481+0000 mon.a (mon.0) 1183 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.860211+0000 mon.c (mon.2) 203 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.861181+0000 mon.c (mon.2) 204 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.873918+0000 mon.a (mon.0) 1184 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-02-18T01:24:18.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:17 smithi105 bash[19734]: audit 2024-02-18T01:24:16.914617+0000 mon.c (mon.2) 205 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.623448+0000 mgr.y (mgr.24475) 304 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: cluster 2024-02-18T01:24:17.639948+0000 mgr.y (mgr.24475) 305 : cluster [DBG] pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.870968+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.871112+0000 mon.a (mon.0) 1186 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:24:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: cluster 2024-02-18T01:24:17.871330+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.873685+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.874744+0000 mon.c (mon.2) 206 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.875261+0000 mon.a (mon.0) 1189 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.915442+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.934896+0000 mon.c (mon.2) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.935261+0000 mon.a (mon.0) 1190 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.936487+0000 mon.c (mon.2) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.936827+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.937383+0000 mon.c (mon.2) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[27669]: audit 2024-02-18T01:24:17.937730+0000 mon.a (mon.0) 1192 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.623448+0000 mgr.y (mgr.24475) 304 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: cluster 2024-02-18T01:24:17.639948+0000 mgr.y (mgr.24475) 305 : cluster [DBG] pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.870968+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.871112+0000 mon.a (mon.0) 1186 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: cluster 2024-02-18T01:24:17.871330+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.873685+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.874744+0000 mon.c (mon.2) 206 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.875261+0000 mon.a (mon.0) 1189 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.915442+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.934896+0000 mon.c (mon.2) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.935261+0000 mon.a (mon.0) 1190 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.936487+0000 mon.c (mon.2) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.936827+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.937383+0000 mon.c (mon.2) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:19.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:18 smithi105 bash[19734]: audit 2024-02-18T01:24:17.937730+0000 mon.a (mon.0) 1192 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.623448+0000 mgr.y (mgr.24475) 304 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: cluster 2024-02-18T01:24:17.639948+0000 mgr.y (mgr.24475) 305 : cluster [DBG] pgmap v274: 356 pgs: 32 creating+peering, 324 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.870968+0000 mon.a (mon.0) 1185 : audit [INF] from='client.? v1:172.21.15.105:0/2624957528' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi105-61649-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.871112+0000 mon.a (mon.0) 1186 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: cluster 2024-02-18T01:24:17.871330+0000 mon.a (mon.0) 1187 : cluster [DBG] osdmap e132: 8 total, 8 up, 8 in 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.873685+0000 mon.a (mon.0) 1188 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.874744+0000 mon.c (mon.2) 206 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.875261+0000 mon.a (mon.0) 1189 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.915442+0000 mon.c (mon.2) 207 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.934896+0000 mon.c (mon.2) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.935261+0000 mon.a (mon.0) 1190 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.936487+0000 mon.c (mon.2) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.936827+0000 mon.a (mon.0) 1191 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.937383+0000 mon.c (mon.2) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:19.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:18 smithi155 bash[20805]: audit 2024-02-18T01:24:17.937730+0000 mon.a (mon.0) 1192 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.875608+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.875765+0000 mon.a (mon.0) 1194 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:24:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.876243+0000 mon.a (mon.0) 1195 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: cluster 2024-02-18T01:24:18.876894+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T01:24:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.879399+0000 mon.c (mon.2) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.891041+0000 mon.a (mon.0) 1197 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.891258+0000 mon.c (mon.2) 212 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.892056+0000 mon.a (mon.0) 1198 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[27669]: audit 2024-02-18T01:24:18.918565+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.875608+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.875765+0000 mon.a (mon.0) 1194 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.876243+0000 mon.a (mon.0) 1195 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: cluster 2024-02-18T01:24:18.876894+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.879399+0000 mon.c (mon.2) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.891041+0000 mon.a (mon.0) 1197 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.891258+0000 mon.c (mon.2) 212 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.892056+0000 mon.a (mon.0) 1198 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:20 smithi105 bash[19734]: audit 2024-02-18T01:24:18.918565+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.875608+0000 mon.a (mon.0) 1193 : audit [INF] from='client.? v1:172.21.15.105:0/2291082740' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.875765+0000 mon.a (mon.0) 1194 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.876243+0000 mon.a (mon.0) 1195 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: cluster 2024-02-18T01:24:18.876894+0000 mon.a (mon.0) 1196 : cluster [DBG] osdmap e133: 8 total, 8 up, 8 in 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.879399+0000 mon.c (mon.2) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.891041+0000 mon.a (mon.0) 1197 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:20.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.891258+0000 mon.c (mon.2) 212 : audit [INF] from='client.? v1:172.21.15.105:0/1675415499' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.892056+0000 mon.a (mon.0) 1198 : audit [INF] from='client.24749 ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]: dispatch 2024-02-18T01:24:20.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:20 smithi155 bash[20805]: audit 2024-02-18T01:24:18.918565+0000 mon.c (mon.2) 213 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: seed 61790 2024-02-18T01:24:20.908 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (9 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (315 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (15 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (12 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-02-18T01:24:20.909 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (3 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (9 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (7 ms) 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: osd_max_attr_size = 0 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-02-18T01:24:20.910 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1730 ms) 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (498 ms) 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: done waiting, doing copies 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: waiting for final deep scrubs... 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: done waiting 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60761 ms) 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (55 ms) 2024-02-18T01:24:20.911 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (2 ms) 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5464 ms) 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-02-18T01:24:20.912 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: cluster 2024-02-18T01:24:19.640866+0000 mgr.y (mgr.24475) 306 : cluster [DBG] pgmap v277: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:19.919630+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.323961+0000 mon.a (mon.0) 1199 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: cluster 2024-02-18T01:24:20.324094+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.326655+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.898574+0000 mon.a (mon.0) 1202 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.899681+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: cluster 2024-02-18T01:24:20.900887+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.921467+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.924739+0000 mon.c (mon.2) 216 : audit [INF] from='client.? v1:172.21.15.105:0/1941012162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.925969+0000 mon.a (mon.0) 1205 : audit [INF] from='client.47816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.943844+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.945221+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[27669]: audit 2024-02-18T01:24:20.945591+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: cluster 2024-02-18T01:24:19.640866+0000 mgr.y (mgr.24475) 306 : cluster [DBG] pgmap v277: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:19.919630+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.323961+0000 mon.a (mon.0) 1199 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:24:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: cluster 2024-02-18T01:24:20.324094+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.326655+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.898574+0000 mon.a (mon.0) 1202 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.899681+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: cluster 2024-02-18T01:24:20.900887+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.921467+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.924739+0000 mon.c (mon.2) 216 : audit [INF] from='client.? v1:172.21.15.105:0/1941012162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.925969+0000 mon.a (mon.0) 1205 : audit [INF] from='client.47816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.943844+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.945221+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:21 smithi105 bash[19734]: audit 2024-02-18T01:24:20.945591+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: cluster 2024-02-18T01:24:19.640866+0000 mgr.y (mgr.24475) 306 : cluster [DBG] pgmap v277: 324 pgs: 32 unknown, 292 active+clean; 216 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:19.919630+0000 mon.c (mon.2) 214 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.323961+0000 mon.a (mon.0) 1199 : audit [INF] from='client.24749 ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi105-61790-1","app":"app1","key":"key1"}]': finished 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: cluster 2024-02-18T01:24:20.324094+0000 mon.a (mon.0) 1200 : cluster [DBG] osdmap e134: 8 total, 8 up, 8 in 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.326655+0000 mon.a (mon.0) 1201 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.898574+0000 mon.a (mon.0) 1202 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi105-61914-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.899681+0000 mon.a (mon.0) 1203 : audit [INF] from='client.? v1:172.21.15.105:0/54000285' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi105-61649-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: cluster 2024-02-18T01:24:20.900887+0000 mon.a (mon.0) 1204 : cluster [DBG] osdmap e135: 8 total, 8 up, 8 in 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.921467+0000 mon.c (mon.2) 215 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.924739+0000 mon.c (mon.2) 216 : audit [INF] from='client.? v1:172.21.15.105:0/1941012162' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.925969+0000 mon.a (mon.0) 1205 : audit [INF] from='client.47816 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.943844+0000 mon.a (mon.0) 1206 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.945221+0000 mon.a (mon.0) 1207 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[20805]: audit 2024-02-18T01:24:20.945591+0000 mon.a (mon.0) 1208 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: audit 2024-02-18T01:24:21.504550+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: cluster 2024-02-18T01:24:21.641786+0000 mgr.y (mgr.24475) 307 : cluster [DBG] pgmap v280: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: audit 2024-02-18T01:24:21.900796+0000 mon.a (mon.0) 1210 : audit [INF] from='client.47816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:22.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: audit 2024-02-18T01:24:21.900991+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: cluster 2024-02-18T01:24:21.901885+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T01:24:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: audit 2024-02-18T01:24:21.907945+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[27669]: audit 2024-02-18T01:24:21.922629+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:22.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: audit 2024-02-18T01:24:21.504550+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:22.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: cluster 2024-02-18T01:24:21.641786+0000 mgr.y (mgr.24475) 307 : cluster [DBG] pgmap v280: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: audit 2024-02-18T01:24:21.900796+0000 mon.a (mon.0) 1210 : audit [INF] from='client.47816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: audit 2024-02-18T01:24:21.900991+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: cluster 2024-02-18T01:24:21.901885+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T01:24:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: audit 2024-02-18T01:24:21.907945+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:22.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:22 smithi105 bash[19734]: audit 2024-02-18T01:24:21.922629+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: audit 2024-02-18T01:24:21.504550+0000 mon.a (mon.0) 1209 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: cluster 2024-02-18T01:24:21.641786+0000 mgr.y (mgr.24475) 307 : cluster [DBG] pgmap v280: 332 pgs: 72 unknown, 260 active+clean; 15 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: audit 2024-02-18T01:24:21.900796+0000 mon.a (mon.0) 1210 : audit [INF] from='client.47816 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi105-61657-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: audit 2024-02-18T01:24:21.900991+0000 mon.a (mon.0) 1211 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61790-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: cluster 2024-02-18T01:24:21.901885+0000 mon.a (mon.0) 1212 : cluster [DBG] osdmap e136: 8 total, 8 up, 8 in 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: audit 2024-02-18T01:24:21.907945+0000 mon.a (mon.0) 1213 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:22 smithi155 bash[20805]: audit 2024-02-18T01:24:21.922629+0000 mon.c (mon.2) 217 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:24.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[27669]: audit 2024-02-18T01:24:22.923868+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:24.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[27669]: cluster 2024-02-18T01:24:23.334242+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T01:24:24.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[27669]: audit 2024-02-18T01:24:23.338591+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[19734]: audit 2024-02-18T01:24:22.923868+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:24.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[19734]: cluster 2024-02-18T01:24:23.334242+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T01:24:24.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:23 smithi105 bash[19734]: audit 2024-02-18T01:24:23.338591+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:23 smithi155 bash[20805]: audit 2024-02-18T01:24:22.923868+0000 mon.c (mon.2) 218 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:23 smithi155 bash[20805]: cluster 2024-02-18T01:24:23.334242+0000 mon.a (mon.0) 1214 : cluster [DBG] osdmap e137: 8 total, 8 up, 8 in 2024-02-18T01:24:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:23 smithi155 bash[20805]: audit 2024-02-18T01:24:23.338591+0000 mon.a (mon.0) 1215 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: cluster 2024-02-18T01:24:23.642644+0000 mgr.y (mgr.24475) 308 : cluster [DBG] pgmap v283: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: cluster 2024-02-18T01:24:23.897044+0000 mon.a (mon.0) 1216 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: audit 2024-02-18T01:24:23.917379+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: audit 2024-02-18T01:24:23.917605+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: cluster 2024-02-18T01:24:23.920419+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T01:24:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: audit 2024-02-18T01:24:23.924812+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: audit 2024-02-18T01:24:23.937909+0000 mon.c (mon.2) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2920705954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[27669]: audit 2024-02-18T01:24:24.368773+0000 mon.a (mon.0) 1220 : audit [INF] from='client.47834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: cluster 2024-02-18T01:24:23.642644+0000 mgr.y (mgr.24475) 308 : cluster [DBG] pgmap v283: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: cluster 2024-02-18T01:24:23.897044+0000 mon.a (mon.0) 1216 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: audit 2024-02-18T01:24:23.917379+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: audit 2024-02-18T01:24:23.917605+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: cluster 2024-02-18T01:24:23.920419+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: audit 2024-02-18T01:24:23.924812+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: audit 2024-02-18T01:24:23.937909+0000 mon.c (mon.2) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2920705954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:24 smithi105 bash[19734]: audit 2024-02-18T01:24:24.368773+0000 mon.a (mon.0) 1220 : audit [INF] from='client.47834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: cluster 2024-02-18T01:24:23.642644+0000 mgr.y (mgr.24475) 308 : cluster [DBG] pgmap v283: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: cluster 2024-02-18T01:24:23.897044+0000 mon.a (mon.0) 1216 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: audit 2024-02-18T01:24:23.917379+0000 mon.a (mon.0) 1217 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61790-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: audit 2024-02-18T01:24:23.917605+0000 mon.a (mon.0) 1218 : audit [INF] from='client.? v1:172.21.15.105:0/795189694' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi105-61649-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: cluster 2024-02-18T01:24:23.920419+0000 mon.a (mon.0) 1219 : cluster [DBG] osdmap e138: 8 total, 8 up, 8 in 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: audit 2024-02-18T01:24:23.924812+0000 mon.c (mon.2) 219 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: audit 2024-02-18T01:24:23.937909+0000 mon.c (mon.2) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2920705954' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:25.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:24 smithi155 bash[20805]: audit 2024-02-18T01:24:24.368773+0000 mon.a (mon.0) 1220 : audit [INF] from='client.47834 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[19734]: audit 2024-02-18T01:24:24.931814+0000 mon.a (mon.0) 1221 : audit [INF] from='client.47834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[19734]: cluster 2024-02-18T01:24:24.932088+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T01:24:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[19734]: audit 2024-02-18T01:24:24.934336+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[19734]: audit 2024-02-18T01:24:24.934459+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[27669]: audit 2024-02-18T01:24:24.931814+0000 mon.a (mon.0) 1221 : audit [INF] from='client.47834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[27669]: cluster 2024-02-18T01:24:24.932088+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T01:24:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[27669]: audit 2024-02-18T01:24:24.934336+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:25 smithi105 bash[27669]: audit 2024-02-18T01:24:24.934459+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:25 smithi155 bash[20805]: audit 2024-02-18T01:24:24.931814+0000 mon.a (mon.0) 1221 : audit [INF] from='client.47834 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi105-61657-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:25 smithi155 bash[20805]: cluster 2024-02-18T01:24:24.932088+0000 mon.a (mon.0) 1222 : cluster [DBG] osdmap e139: 8 total, 8 up, 8 in 2024-02-18T01:24:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:25 smithi155 bash[20805]: audit 2024-02-18T01:24:24.934336+0000 mon.c (mon.2) 221 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:25 smithi155 bash[20805]: audit 2024-02-18T01:24:24.934459+0000 mon.a (mon.0) 1223 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:26.959 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:26] "GET /metrics HTTP/1.1" 200 194396 "" "Prometheus/2.33.4" 2024-02-18T01:24:26.960 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: cluster 2024-02-18T01:24:25.643608+0000 mgr.y (mgr.24475) 309 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: audit 2024-02-18T01:24:25.936513+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: cluster 2024-02-18T01:24:25.936807+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T01:24:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: audit 2024-02-18T01:24:25.938647+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: audit 2024-02-18T01:24:25.946681+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[19734]: audit 2024-02-18T01:24:25.947811+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: cluster 2024-02-18T01:24:25.643608+0000 mgr.y (mgr.24475) 309 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: audit 2024-02-18T01:24:25.936513+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: cluster 2024-02-18T01:24:25.936807+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: audit 2024-02-18T01:24:25.938647+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: audit 2024-02-18T01:24:25.946681+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:26 smithi105 bash[27669]: audit 2024-02-18T01:24:25.947811+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: cluster 2024-02-18T01:24:25.643608+0000 mgr.y (mgr.24475) 309 : cluster [DBG] pgmap v286: 340 pgs: 72 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: audit 2024-02-18T01:24:25.936513+0000 mon.a (mon.0) 1224 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: cluster 2024-02-18T01:24:25.936807+0000 mon.a (mon.0) 1225 : cluster [DBG] osdmap e140: 8 total, 8 up, 8 in 2024-02-18T01:24:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: audit 2024-02-18T01:24:25.938647+0000 mon.c (mon.2) 222 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: audit 2024-02-18T01:24:25.946681+0000 mon.a (mon.0) 1226 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:26 smithi155 bash[20805]: audit 2024-02-18T01:24:25.947811+0000 mon.a (mon.0) 1227 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.965 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:26.944263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:27.965 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:26.950801+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:26.951153+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: cluster 2024-02-18T01:24:26.951412+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:26.953403+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.105:0/3295466820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:26.956491+0000 mon.a (mon.0) 1231 : audit [INF] from='client.47846 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:27.943643+0000 mon.a (mon.0) 1232 : audit [INF] from='client.47846 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: cluster 2024-02-18T01:24:27.944023+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T01:24:27.966 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:27 smithi155 bash[20805]: audit 2024-02-18T01:24:27.945676+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:26.944263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:26.950801+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:26.951153+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: cluster 2024-02-18T01:24:26.951412+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:26.953403+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.105:0/3295466820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:26.956491+0000 mon.a (mon.0) 1231 : audit [INF] from='client.47846 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:27.943643+0000 mon.a (mon.0) 1232 : audit [INF] from='client.47846 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: cluster 2024-02-18T01:24:27.944023+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[27669]: audit 2024-02-18T01:24:27.945676+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:26.944263+0000 mon.c (mon.2) 223 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:26.950801+0000 mon.a (mon.0) 1228 : audit [INF] from='client.? v1:172.21.15.105:0/1134416247' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi105-61649-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:26.951153+0000 mon.a (mon.0) 1229 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61790-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: cluster 2024-02-18T01:24:26.951412+0000 mon.a (mon.0) 1230 : cluster [DBG] osdmap e141: 8 total, 8 up, 8 in 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:26.953403+0000 mon.c (mon.2) 224 : audit [INF] from='client.? v1:172.21.15.105:0/3295466820' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:26.956491+0000 mon.a (mon.0) 1231 : audit [INF] from='client.47846 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:27.943643+0000 mon.a (mon.0) 1232 : audit [INF] from='client.47846 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi105-61657-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: cluster 2024-02-18T01:24:27.944023+0000 mon.a (mon.0) 1233 : cluster [DBG] osdmap e142: 8 total, 8 up, 8 in 2024-02-18T01:24:28.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:27 smithi105 bash[19734]: audit 2024-02-18T01:24:27.945676+0000 mon.c (mon.2) 225 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: audit 2024-02-18T01:24:27.627205+0000 mgr.y (mgr.24475) 310 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: cluster 2024-02-18T01:24:27.646708+0000 mgr.y (mgr.24475) 311 : cluster [DBG] pgmap v289: 372 pgs: 32 creating+peering, 340 active+clean; 16 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-02-18T01:24:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: cluster 2024-02-18T01:24:28.803984+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: audit 2024-02-18T01:24:28.805178+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.105:0/1163568509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: audit 2024-02-18T01:24:28.805588+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[19734]: audit 2024-02-18T01:24:28.811320+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48595 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: audit 2024-02-18T01:24:27.627205+0000 mgr.y (mgr.24475) 310 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: cluster 2024-02-18T01:24:27.646708+0000 mgr.y (mgr.24475) 311 : cluster [DBG] pgmap v289: 372 pgs: 32 creating+peering, 340 active+clean; 16 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: cluster 2024-02-18T01:24:28.803984+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: audit 2024-02-18T01:24:28.805178+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.105:0/1163568509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: audit 2024-02-18T01:24:28.805588+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:29 smithi105 bash[27669]: audit 2024-02-18T01:24:28.811320+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48595 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: audit 2024-02-18T01:24:27.627205+0000 mgr.y (mgr.24475) 310 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: cluster 2024-02-18T01:24:27.646708+0000 mgr.y (mgr.24475) 311 : cluster [DBG] pgmap v289: 372 pgs: 32 creating+peering, 340 active+clean; 16 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 MiB/s wr, 4 op/s 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: cluster 2024-02-18T01:24:28.803984+0000 mon.a (mon.0) 1234 : cluster [DBG] osdmap e143: 8 total, 8 up, 8 in 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: audit 2024-02-18T01:24:28.805178+0000 mon.b (mon.1) 134 : audit [INF] from='client.? v1:172.21.15.105:0/1163568509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: audit 2024-02-18T01:24:28.805588+0000 mon.a (mon.0) 1235 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:29 smithi155 bash[20805]: audit 2024-02-18T01:24:28.811320+0000 mon.a (mon.0) 1236 : audit [INF] from='client.48595 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:28.946821+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: cluster 2024-02-18T01:24:29.647648+0000 mgr.y (mgr.24475) 312 : cluster [DBG] pgmap v292: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:29.801618+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:29.801899+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48595 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: cluster 2024-02-18T01:24:29.802188+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:29.809674+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:29.816062+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[19734]: audit 2024-02-18T01:24:29.948384+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:28.946821+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: cluster 2024-02-18T01:24:29.647648+0000 mgr.y (mgr.24475) 312 : cluster [DBG] pgmap v292: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:29.801618+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:30.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:29.801899+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48595 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:30.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: cluster 2024-02-18T01:24:29.802188+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T01:24:30.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:29.809674+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:30.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:29.816062+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:30.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:30 smithi105 bash[27669]: audit 2024-02-18T01:24:29.948384+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:28.946821+0000 mon.c (mon.2) 226 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: cluster 2024-02-18T01:24:29.647648+0000 mgr.y (mgr.24475) 312 : cluster [DBG] pgmap v292: 300 pgs: 32 unknown, 268 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:29.801618+0000 mon.a (mon.0) 1237 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:29.801899+0000 mon.a (mon.0) 1238 : audit [INF] from='client.48595 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi105-61649-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: cluster 2024-02-18T01:24:29.802188+0000 mon.a (mon.0) 1239 : cluster [DBG] osdmap e144: 8 total, 8 up, 8 in 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:29.809674+0000 mon.a (mon.0) 1240 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]: dispatch 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:29.816062+0000 mon.a (mon.0) 1241 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:30.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:30 smithi155 bash[20805]: audit 2024-02-18T01:24:29.948384+0000 mon.c (mon.2) 227 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.810157+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.810533+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: cluster 2024-02-18T01:24:30.810700+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.813722+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.814740+0000 mon.a (mon.0) 1245 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.854853+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.855820+0000 mon.a (mon.0) 1246 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.856730+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.857031+0000 mon.a (mon.0) 1247 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.857563+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:30.949964+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[19734]: audit 2024-02-18T01:24:31.284865+0000 mon.a (mon.0) 1248 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.810157+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.810533+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: cluster 2024-02-18T01:24:30.810700+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.813722+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.814740+0000 mon.a (mon.0) 1245 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.854853+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.855820+0000 mon.a (mon.0) 1246 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.856730+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.857031+0000 mon.a (mon.0) 1247 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.857563+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:30.949964+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:32.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:31 smithi105 bash[27669]: audit 2024-02-18T01:24:31.284865+0000 mon.a (mon.0) 1248 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.810157+0000 mon.a (mon.0) 1242 : audit [INF] from='client.? v1:172.21.15.105:0/766829613' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61790-24"}]': finished 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.810533+0000 mon.a (mon.0) 1243 : audit [INF] from='client.? v1:172.21.15.105:0/2030886181' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi105-61657-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: cluster 2024-02-18T01:24:30.810700+0000 mon.a (mon.0) 1244 : cluster [DBG] osdmap e145: 8 total, 8 up, 8 in 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.813722+0000 mon.c (mon.2) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.814740+0000 mon.a (mon.0) 1245 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.854853+0000 mon.b (mon.1) 135 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.855820+0000 mon.a (mon.0) 1246 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.856730+0000 mon.b (mon.1) 136 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.857031+0000 mon.a (mon.0) 1247 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.857563+0000 mon.b (mon.1) 137 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:30.949964+0000 mon.c (mon.2) 229 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:32.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:31 smithi155 bash[20805]: audit 2024-02-18T01:24:31.284865+0000 mon.a (mon.0) 1248 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots_pp: Running main() from gmock_main.cc 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: seed 61914 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (3500 ms) 2024-02-18T01:24:32.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (4866 ms) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (2008 ms) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (1372 ms) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4363 ms) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (16110 ms total) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4317 ms) 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-02-18T01:24:32.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4560 ms) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (7256 ms) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4209 ms) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2132 ms) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi105-61914-7 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: waiting for snaps to purge 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17982 ms) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (40456 ms total) 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-02-18T01:24:32.833 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (8 ms) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5098 ms) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5106 ms total) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3014 ms) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2004 ms) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2007 ms) 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-02-18T01:24:32.834 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1863 ms) 2024-02-18T01:24:32.835 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8888 ms total) 2024-02-18T01:24:32.835 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: 2024-02-18T01:24:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: cluster 2024-02-18T01:24:31.648520+0000 mgr.y (mgr.24475) 313 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: cluster 2024-02-18T01:24:31.803190+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.814876+0000 mon.a (mon.0) 1250 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.815140+0000 mon.a (mon.0) 1251 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: cluster 2024-02-18T01:24:31.815267+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T01:24:33.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.816742+0000 mon.c (mon.2) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.818596+0000 mon.a (mon.0) 1253 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.823882+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.824867+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.835760+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[27669]: audit 2024-02-18T01:24:31.951320+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:33.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: cluster 2024-02-18T01:24:31.648520+0000 mgr.y (mgr.24475) 313 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: cluster 2024-02-18T01:24:31.803190+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.814876+0000 mon.a (mon.0) 1250 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.815140+0000 mon.a (mon.0) 1251 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: cluster 2024-02-18T01:24:31.815267+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.816742+0000 mon.c (mon.2) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.818596+0000 mon.a (mon.0) 1253 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.493 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.823882+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:33.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.824867+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.835760+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.494 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:33 smithi105 bash[19734]: audit 2024-02-18T01:24:31.951320+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:33.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: cluster 2024-02-18T01:24:31.648520+0000 mgr.y (mgr.24475) 313 : cluster [DBG] pgmap v295: 292 pgs: 32 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:33.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: cluster 2024-02-18T01:24:31.803190+0000 mon.a (mon.0) 1249 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.814876+0000 mon.a (mon.0) 1250 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.815140+0000 mon.a (mon.0) 1251 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi105-61649-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: cluster 2024-02-18T01:24:31.815267+0000 mon.a (mon.0) 1252 : cluster [DBG] osdmap e146: 8 total, 8 up, 8 in 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.816742+0000 mon.c (mon.2) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2526229652' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.818596+0000 mon.a (mon.0) 1253 : audit [INF] from='client.47804 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]: dispatch 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.823882+0000 mon.a (mon.0) 1254 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:33.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.824867+0000 mon.b (mon.1) 138 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.835760+0000 mon.a (mon.0) 1255 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:33.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:33 smithi155 bash[20805]: audit 2024-02-18T01:24:31.951320+0000 mon.c (mon.2) 231 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:32.817880+0000 mon.a (mon.0) 1256 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:32.818050+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:32.854650+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:32.952678+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: cluster 2024-02-18T01:24:33.232791+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.234768+0000 mon.c (mon.2) 233 : audit [INF] from='client.? v1:172.21.15.105:0/598727994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.243293+0000 mon.a (mon.0) 1259 : audit [INF] from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.244274+0000 mon.a (mon.0) 1260 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.244953+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.245261+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.245740+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.245998+0000 mon.a (mon.0) 1262 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.821456+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:34.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.822065+0000 mon.a (mon.0) 1264 : audit [INF] from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:32.817880+0000 mon.a (mon.0) 1256 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:32.818050+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:32.854650+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:32.952678+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: cluster 2024-02-18T01:24:33.232791+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.234768+0000 mon.c (mon.2) 233 : audit [INF] from='client.? v1:172.21.15.105:0/598727994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.243293+0000 mon.a (mon.0) 1259 : audit [INF] from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.244274+0000 mon.a (mon.0) 1260 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.244953+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.245261+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.245740+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.245998+0000 mon.a (mon.0) 1262 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.821456+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.822065+0000 mon.a (mon.0) 1264 : audit [INF] from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.822502+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: cluster 2024-02-18T01:24:33.822795+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.824364+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[19734]: audit 2024-02-18T01:24:33.829824+0000 mon.a (mon.0) 1267 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.822502+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: cluster 2024-02-18T01:24:33.822795+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T01:24:34.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.824364+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:34 smithi105 bash[27669]: audit 2024-02-18T01:24:33.829824+0000 mon.a (mon.0) 1267 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:32.817880+0000 mon.a (mon.0) 1256 : audit [INF] from='client.47804 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi105-61914-15"}]': finished 2024-02-18T01:24:34.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:32.818050+0000 mon.a (mon.0) 1257 : audit [INF] from='client.? v1:172.21.15.105:0/1986792928' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi105-61790-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:32.854650+0000 mon.b (mon.1) 139 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:32.952678+0000 mon.c (mon.2) 232 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: cluster 2024-02-18T01:24:33.232791+0000 mon.a (mon.0) 1258 : cluster [DBG] osdmap e147: 8 total, 8 up, 8 in 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.234768+0000 mon.c (mon.2) 233 : audit [INF] from='client.? v1:172.21.15.105:0/598727994' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.243293+0000 mon.a (mon.0) 1259 : audit [INF] from='client.47873 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.244274+0000 mon.a (mon.0) 1260 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.244953+0000 mon.b (mon.1) 140 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.245261+0000 mon.a (mon.0) 1261 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.245740+0000 mon.b (mon.1) 141 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.245998+0000 mon.a (mon.0) 1262 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.821456+0000 mon.a (mon.0) 1263 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi105-61649-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.822065+0000 mon.a (mon.0) 1264 : audit [INF] from='client.47873 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi105-61657-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.822502+0000 mon.a (mon.0) 1265 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:34.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: cluster 2024-02-18T01:24:33.822795+0000 mon.a (mon.0) 1266 : cluster [DBG] osdmap e148: 8 total, 8 up, 8 in 2024-02-18T01:24:34.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.824364+0000 mon.b (mon.1) 142 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:34.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:34 smithi155 bash[20805]: audit 2024-02-18T01:24:33.829824+0000 mon.a (mon.0) 1267 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: cluster 2024-02-18T01:24:33.649501+0000 mgr.y (mgr.24475) 314 : cluster [DBG] pgmap v298: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:24:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: audit 2024-02-18T01:24:33.954143+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: audit 2024-02-18T01:24:34.955662+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: cluster 2024-02-18T01:24:35.255060+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: audit 2024-02-18T01:24:35.256458+0000 mon.c (mon.2) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2973341204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[19734]: audit 2024-02-18T01:24:35.261567+0000 mon.a (mon.0) 1269 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: cluster 2024-02-18T01:24:33.649501+0000 mgr.y (mgr.24475) 314 : cluster [DBG] pgmap v298: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: audit 2024-02-18T01:24:33.954143+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: audit 2024-02-18T01:24:34.955662+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: cluster 2024-02-18T01:24:35.255060+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: audit 2024-02-18T01:24:35.256458+0000 mon.c (mon.2) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2973341204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:35.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:35 smithi105 bash[27669]: audit 2024-02-18T01:24:35.261567+0000 mon.a (mon.0) 1269 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: cluster 2024-02-18T01:24:33.649501+0000 mgr.y (mgr.24475) 314 : cluster [DBG] pgmap v298: 324 pgs: 32 unknown, 32 creating+peering, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: audit 2024-02-18T01:24:33.954143+0000 mon.c (mon.2) 234 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: audit 2024-02-18T01:24:34.955662+0000 mon.c (mon.2) 235 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: cluster 2024-02-18T01:24:35.255060+0000 mon.a (mon.0) 1268 : cluster [DBG] osdmap e149: 8 total, 8 up, 8 in 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: audit 2024-02-18T01:24:35.256458+0000 mon.c (mon.2) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2973341204' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:35.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:35 smithi155 bash[20805]: audit 2024-02-18T01:24:35.261567+0000 mon.a (mon.0) 1269 : audit [INF] from='client.47885 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:36] "GET /metrics HTTP/1.1" 200 194832 "" "Prometheus/2.33.4" 2024-02-18T01:24:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: cluster 2024-02-18T01:24:35.650377+0000 mgr.y (mgr.24475) 315 : cluster [DBG] pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.838071+0000 mon.a (mon.0) 1270 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.838688+0000 mon.a (mon.0) 1271 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: cluster 2024-02-18T01:24:35.838871+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.843050+0000 mon.c (mon.2) 237 : audit [INF] from='client.? v1:172.21.15.105:0/964395946' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.844481+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.845219+0000 mon.a (mon.0) 1273 : audit [INF] from='client.47891 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.845806+0000 mon.a (mon.0) 1274 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[27669]: audit 2024-02-18T01:24:35.956994+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: cluster 2024-02-18T01:24:35.650377+0000 mgr.y (mgr.24475) 315 : cluster [DBG] pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.838071+0000 mon.a (mon.0) 1270 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.838688+0000 mon.a (mon.0) 1271 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: cluster 2024-02-18T01:24:35.838871+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.843050+0000 mon.c (mon.2) 237 : audit [INF] from='client.? v1:172.21.15.105:0/964395946' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.844481+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.845219+0000 mon.a (mon.0) 1273 : audit [INF] from='client.47891 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.845806+0000 mon.a (mon.0) 1274 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:36.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:36 smithi105 bash[19734]: audit 2024-02-18T01:24:35.956994+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: cluster 2024-02-18T01:24:35.650377+0000 mgr.y (mgr.24475) 315 : cluster [DBG] pgmap v301: 300 pgs: 40 unknown, 260 active+clean; 15 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.838071+0000 mon.a (mon.0) 1270 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.838688+0000 mon.a (mon.0) 1271 : audit [INF] from='client.47885 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi105-61790-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: cluster 2024-02-18T01:24:35.838871+0000 mon.a (mon.0) 1272 : cluster [DBG] osdmap e150: 8 total, 8 up, 8 in 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.843050+0000 mon.c (mon.2) 237 : audit [INF] from='client.? v1:172.21.15.105:0/964395946' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.844481+0000 mon.b (mon.1) 143 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.845219+0000 mon.a (mon.0) 1273 : audit [INF] from='client.47891 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.845806+0000 mon.a (mon.0) 1274 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:37.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:36 smithi155 bash[20805]: audit 2024-02-18T01:24:35.956994+0000 mon.c (mon.2) 238 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:37.866 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp api_aio: Running main() from gmock_main.cc 2024-02-18T01:24:37.866 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] Global test environment set-up. 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.TooBig (11501 ms) 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (6488 ms) 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (3870 ms) 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2812 ms) 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (2926 ms) 2024-02-18T01:24:37.867 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3907 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3107 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3169 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3597 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3086 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3147 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (4216 ms) 2024-02-18T01:24:37.868 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3103 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (3014 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.Flush (2910 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (2991 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3021 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2984 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3018 ms) 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-02-18T01:24:37.869 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3012 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3053 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (3030 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3034 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3004 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2866 ms) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] 25 tests from LibRadosAio (90866 ms total) 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-02-18T01:24:37.870 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7047 ms) 2024-02-18T01:24:37.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:36.844308+0000 mon.a (mon.0) 1275 : audit [INF] from='client.47891 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:36.844440+0000 mon.a (mon.0) 1276 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: cluster 2024-02-18T01:24:36.844632+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:36.850437+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:36.851294+0000 mon.a (mon.0) 1278 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:36.958500+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:37.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:37 smithi155 bash[20805]: audit 2024-02-18T01:24:37.655927+0000 mon.a (mon.0) 1279 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:36.844308+0000 mon.a (mon.0) 1275 : audit [INF] from='client.47891 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:36.844440+0000 mon.a (mon.0) 1276 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: cluster 2024-02-18T01:24:36.844632+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:36.850437+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:36.851294+0000 mon.a (mon.0) 1278 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:36.958500+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[19734]: audit 2024-02-18T01:24:37.655927+0000 mon.a (mon.0) 1279 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:36.844308+0000 mon.a (mon.0) 1275 : audit [INF] from='client.47891 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi105-61657-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:36.844440+0000 mon.a (mon.0) 1276 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: cluster 2024-02-18T01:24:36.844632+0000 mon.a (mon.0) 1277 : cluster [DBG] osdmap e151: 8 total, 8 up, 8 in 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:36.850437+0000 mon.b (mon.1) 144 : audit [INF] from='client.? v1:172.21.15.105:0/131957724' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:36.851294+0000 mon.a (mon.0) 1278 : audit [INF] from='client.48607 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]: dispatch 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:36.958500+0000 mon.c (mon.2) 239 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:37 smithi105 bash[27669]: audit 2024-02-18T01:24:37.655927+0000 mon.a (mon.0) 1279 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.638548+0000 mgr.y (mgr.24475) 316 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: cluster 2024-02-18T01:24:37.652949+0000 mgr.y (mgr.24475) 317 : cluster [DBG] pgmap v304: 300 pgs: 300 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: cluster 2024-02-18T01:24:37.842539+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.853041+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.853309+0000 mon.a (mon.0) 1282 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: cluster 2024-02-18T01:24:37.855541+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.869261+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.105:0/163454133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.877517+0000 mon.a (mon.0) 1284 : audit [INF] from='client.48640 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.894262+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.897680+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.898684+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:37.959825+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:38.859441+0000 mon.a (mon.0) 1288 : audit [INF] from='client.48640 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:38.859645+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: cluster 2024-02-18T01:24:38.859755+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:38.863539+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[27669]: audit 2024-02-18T01:24:38.864303+0000 mon.a (mon.0) 1292 : audit [INF] from='client.47906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.638548+0000 mgr.y (mgr.24475) 316 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: cluster 2024-02-18T01:24:37.652949+0000 mgr.y (mgr.24475) 317 : cluster [DBG] pgmap v304: 300 pgs: 300 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: cluster 2024-02-18T01:24:37.842539+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.853041+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.853309+0000 mon.a (mon.0) 1282 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-18T01:24:39.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: cluster 2024-02-18T01:24:37.855541+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.869261+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.105:0/163454133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.877517+0000 mon.a (mon.0) 1284 : audit [INF] from='client.48640 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.894262+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.897680+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.898684+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:37.959825+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:38.859441+0000 mon.a (mon.0) 1288 : audit [INF] from='client.48640 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:38.859645+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: cluster 2024-02-18T01:24:38.859755+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:38.863539+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:38 smithi105 bash[19734]: audit 2024-02-18T01:24:38.864303+0000 mon.a (mon.0) 1292 : audit [INF] from='client.47906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.638548+0000 mgr.y (mgr.24475) 316 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: cluster 2024-02-18T01:24:37.652949+0000 mgr.y (mgr.24475) 317 : cluster [DBG] pgmap v304: 300 pgs: 300 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: cluster 2024-02-18T01:24:37.842539+0000 mon.a (mon.0) 1280 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.853041+0000 mon.a (mon.0) 1281 : audit [INF] from='client.48607 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi105-61649-26"}]': finished 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.853309+0000 mon.a (mon.0) 1282 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: cluster 2024-02-18T01:24:37.855541+0000 mon.a (mon.0) 1283 : cluster [DBG] osdmap e152: 8 total, 8 up, 8 in 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.869261+0000 mon.b (mon.1) 145 : audit [INF] from='client.? v1:172.21.15.105:0/163454133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.877517+0000 mon.a (mon.0) 1284 : audit [INF] from='client.48640 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.894262+0000 mon.a (mon.0) 1285 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.897680+0000 mon.a (mon.0) 1286 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.898684+0000 mon.a (mon.0) 1287 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:37.959825+0000 mon.c (mon.2) 240 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:38.859441+0000 mon.a (mon.0) 1288 : audit [INF] from='client.48640 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi105-61790-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:38.859645+0000 mon.a (mon.0) 1289 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi105-61649-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: cluster 2024-02-18T01:24:38.859755+0000 mon.a (mon.0) 1290 : cluster [DBG] osdmap e153: 8 total, 8 up, 8 in 2024-02-18T01:24:39.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:38.863539+0000 mon.a (mon.0) 1291 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:39.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:38 smithi155 bash[20805]: audit 2024-02-18T01:24:38.864303+0000 mon.a (mon.0) 1292 : audit [INF] from='client.47906 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[19734]: audit 2024-02-18T01:24:38.862689+0000 mon.c (mon.2) 241 : audit [INF] from='client.? v1:172.21.15.105:0/751364082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[19734]: audit 2024-02-18T01:24:38.954006+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[19734]: audit 2024-02-18T01:24:38.954533+0000 mon.c (mon.2) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[19734]: audit 2024-02-18T01:24:38.954819+0000 mon.a (mon.0) 1293 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[27669]: audit 2024-02-18T01:24:38.862689+0000 mon.c (mon.2) 241 : audit [INF] from='client.? v1:172.21.15.105:0/751364082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[27669]: audit 2024-02-18T01:24:38.954006+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[27669]: audit 2024-02-18T01:24:38.954533+0000 mon.c (mon.2) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:40 smithi105 bash[27669]: audit 2024-02-18T01:24:38.954819+0000 mon.a (mon.0) 1293 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:40 smithi155 bash[20805]: audit 2024-02-18T01:24:38.862689+0000 mon.c (mon.2) 241 : audit [INF] from='client.? v1:172.21.15.105:0/751364082' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:40 smithi155 bash[20805]: audit 2024-02-18T01:24:38.954006+0000 mon.c (mon.2) 242 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:40 smithi155 bash[20805]: audit 2024-02-18T01:24:38.954533+0000 mon.c (mon.2) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:40 smithi155 bash[20805]: audit 2024-02-18T01:24:38.954819+0000 mon.a (mon.0) 1293 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]: dispatch 2024-02-18T01:24:41.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: cluster 2024-02-18T01:24:39.654009+0000 mgr.y (mgr.24475) 318 : cluster [DBG] pgmap v307: 332 pgs: 64 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.302596+0000 mon.a (mon.0) 1294 : audit [INF] from='client.47906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.303550+0000 mon.a (mon.0) 1295 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: cluster 2024-02-18T01:24:40.303821+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.304668+0000 mon.c (mon.2) 244 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.306112+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.309571+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.318244+0000 mon.a (mon.0) 1297 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.322573+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.323803+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.324073+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.866873+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:41.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.867054+0000 mon.a (mon.0) 1302 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.867597+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: cluster 2024-02-18T01:24:40.867816+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.870056+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[19734]: audit 2024-02-18T01:24:40.871418+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: cluster 2024-02-18T01:24:39.654009+0000 mgr.y (mgr.24475) 318 : cluster [DBG] pgmap v307: 332 pgs: 64 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.302596+0000 mon.a (mon.0) 1294 : audit [INF] from='client.47906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.303550+0000 mon.a (mon.0) 1295 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: cluster 2024-02-18T01:24:40.303821+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.304668+0000 mon.c (mon.2) 244 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.306112+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.309571+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.318244+0000 mon.a (mon.0) 1297 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.322573+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.323803+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.324073+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.866873+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.867054+0000 mon.a (mon.0) 1302 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.867597+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: cluster 2024-02-18T01:24:40.867816+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.870056+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:41 smithi105 bash[27669]: audit 2024-02-18T01:24:40.871418+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: cluster 2024-02-18T01:24:39.654009+0000 mgr.y (mgr.24475) 318 : cluster [DBG] pgmap v307: 332 pgs: 64 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:24:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.302596+0000 mon.a (mon.0) 1294 : audit [INF] from='client.47906 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi105-61657-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.303550+0000 mon.a (mon.0) 1295 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pg_num","val":"11"}]': finished 2024-02-18T01:24:41.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: cluster 2024-02-18T01:24:40.303821+0000 mon.a (mon.0) 1296 : cluster [DBG] osdmap e154: 8 total, 8 up, 8 in 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.304668+0000 mon.c (mon.2) 244 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.306112+0000 mon.c (mon.2) 245 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.309571+0000 mon.c (mon.2) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.318244+0000 mon.a (mon.0) 1297 : audit [INF] from='client.24731 ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.322573+0000 mon.a (mon.0) 1298 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.323803+0000 mon.a (mon.0) 1299 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.324073+0000 mon.a (mon.0) 1300 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.866873+0000 mon.a (mon.0) 1301 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi105-61649-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.867054+0000 mon.a (mon.0) 1302 : audit [INF] from='client.24731 ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi105-61689-1","var":"pgp_num","val":"11"}]': finished 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.867597+0000 mon.a (mon.0) 1303 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: cluster 2024-02-18T01:24:40.867816+0000 mon.a (mon.0) 1304 : cluster [DBG] osdmap e155: 8 total, 8 up, 8 in 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.870056+0000 mon.c (mon.2) 247 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:41.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[20805]: audit 2024-02-18T01:24:40.871418+0000 mon.a (mon.0) 1305 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:42.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:41.054703+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:41.341943+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: cluster 2024-02-18T01:24:41.654934+0000 mgr.y (mgr.24475) 319 : cluster [DBG] pgmap v310: 276 pgs: 8 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:41.874372+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:42.291330+0000 mon.a (mon.0) 1307 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: cluster 2024-02-18T01:24:42.291585+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:42.292337+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.105:0/3687481620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:42.292820+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:42.297727+0000 mon.a (mon.0) 1309 : audit [INF] from='client.47915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[27669]: audit 2024-02-18T01:24:42.299181+0000 mon.a (mon.0) 1310 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:42.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:41.054703+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:41.341943+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: cluster 2024-02-18T01:24:41.654934+0000 mgr.y (mgr.24475) 319 : cluster [DBG] pgmap v310: 276 pgs: 8 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:41.874372+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:42.291330+0000 mon.a (mon.0) 1307 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: cluster 2024-02-18T01:24:42.291585+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:42.292337+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.105:0/3687481620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:42.292820+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:42.297727+0000 mon.a (mon.0) 1309 : audit [INF] from='client.47915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:42 smithi105 bash[19734]: audit 2024-02-18T01:24:42.299181+0000 mon.a (mon.0) 1310 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:41.054703+0000 mon.b (mon.1) 146 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:41.341943+0000 mon.a (mon.0) 1306 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: cluster 2024-02-18T01:24:41.654934+0000 mgr.y (mgr.24475) 319 : cluster [DBG] pgmap v310: 276 pgs: 8 unknown, 268 active+clean; 23 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:41.874372+0000 mon.c (mon.2) 248 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:42.291330+0000 mon.a (mon.0) 1307 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: cluster 2024-02-18T01:24:42.291585+0000 mon.a (mon.0) 1308 : cluster [DBG] osdmap e156: 8 total, 8 up, 8 in 2024-02-18T01:24:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:42.292337+0000 mon.c (mon.2) 249 : audit [INF] from='client.? v1:172.21.15.105:0/3687481620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:42.292820+0000 mon.b (mon.1) 147 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:42.297727+0000 mon.a (mon.0) 1309 : audit [INF] from='client.47915 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:42.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:42 smithi155 bash[20805]: audit 2024-02-18T01:24:42.299181+0000 mon.a (mon.0) 1310 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]: dispatch 2024-02-18T01:24:43.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: cluster 2024-02-18T01:24:42.863840+0000 mon.a (mon.0) 1311 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:43.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: audit 2024-02-18T01:24:42.873814+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:43.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: audit 2024-02-18T01:24:42.874061+0000 mon.a (mon.0) 1313 : audit [INF] from='client.47915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:43.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: audit 2024-02-18T01:24:42.874167+0000 mon.a (mon.0) 1314 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: cluster 2024-02-18T01:24:42.874259+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: audit 2024-02-18T01:24:42.877367+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[19734]: audit 2024-02-18T01:24:42.880615+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: cluster 2024-02-18T01:24:42.863840+0000 mon.a (mon.0) 1311 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: audit 2024-02-18T01:24:42.873814+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: audit 2024-02-18T01:24:42.874061+0000 mon.a (mon.0) 1313 : audit [INF] from='client.47915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: audit 2024-02-18T01:24:42.874167+0000 mon.a (mon.0) 1314 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: cluster 2024-02-18T01:24:42.874259+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: audit 2024-02-18T01:24:42.877367+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:43 smithi105 bash[27669]: audit 2024-02-18T01:24:42.880615+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: cluster 2024-02-18T01:24:42.863840+0000 mon.a (mon.0) 1311 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: audit 2024-02-18T01:24:42.873814+0000 mon.a (mon.0) 1312 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi105-61790-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: audit 2024-02-18T01:24:42.874061+0000 mon.a (mon.0) 1313 : audit [INF] from='client.47915 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi105-61657-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: audit 2024-02-18T01:24:42.874167+0000 mon.a (mon.0) 1314 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-13"}]': finished 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: cluster 2024-02-18T01:24:42.874259+0000 mon.a (mon.0) 1315 : cluster [DBG] osdmap e157: 8 total, 8 up, 8 in 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: audit 2024-02-18T01:24:42.877367+0000 mon.c (mon.2) 250 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:43 smithi155 bash[20805]: audit 2024-02-18T01:24:42.880615+0000 mon.a (mon.0) 1316 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: cluster 2024-02-18T01:24:43.655925+0000 mgr.y (mgr.24475) 320 : cluster [DBG] pgmap v313: 308 pgs: 32 creating+peering, 8 unknown, 268 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 14 KiB/s wr, 32 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: audit 2024-02-18T01:24:43.658930+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: audit 2024-02-18T01:24:43.880048+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: audit 2024-02-18T01:24:43.880298+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: audit 2024-02-18T01:24:43.881788+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: cluster 2024-02-18T01:24:43.882082+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[19734]: audit 2024-02-18T01:24:43.887567+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: cluster 2024-02-18T01:24:43.655925+0000 mgr.y (mgr.24475) 320 : cluster [DBG] pgmap v313: 308 pgs: 32 creating+peering, 8 unknown, 268 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 14 KiB/s wr, 32 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: audit 2024-02-18T01:24:43.658930+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: audit 2024-02-18T01:24:43.880048+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: audit 2024-02-18T01:24:43.880298+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: audit 2024-02-18T01:24:43.881788+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T01:24:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: cluster 2024-02-18T01:24:43.882082+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-18T01:24:45.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:44 smithi105 bash[27669]: audit 2024-02-18T01:24:43.887567+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: cluster 2024-02-18T01:24:43.655925+0000 mgr.y (mgr.24475) 320 : cluster [DBG] pgmap v313: 308 pgs: 32 creating+peering, 8 unknown, 268 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 9.2 KiB/s rd, 14 KiB/s wr, 32 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: audit 2024-02-18T01:24:43.658930+0000 mon.a (mon.0) 1317 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: audit 2024-02-18T01:24:43.880048+0000 mon.c (mon.2) 251 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: audit 2024-02-18T01:24:43.880298+0000 mon.a (mon.0) 1318 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: audit 2024-02-18T01:24:43.881788+0000 mon.a (mon.0) 1319 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: cluster 2024-02-18T01:24:43.882082+0000 mon.a (mon.0) 1320 : cluster [DBG] osdmap e158: 8 total, 8 up, 8 in 2024-02-18T01:24:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:44 smithi155 bash[20805]: audit 2024-02-18T01:24:43.887567+0000 mon.a (mon.0) 1321 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]: dispatch 2024-02-18T01:24:46.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.877300+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:46.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: cluster 2024-02-18T01:24:44.877465+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.879679+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.881540+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.882154+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1893728271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.882970+0000 mon.c (mon.2) 252 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.887592+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.888438+0000 mon.a (mon.0) 1326 : audit [INF] from='client.48667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.905617+0000 mon.c (mon.2) 253 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.905899+0000 mon.a (mon.0) 1327 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.906847+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.907099+0000 mon.a (mon.0) 1328 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.907514+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[27669]: audit 2024-02-18T01:24:44.907773+0000 mon.a (mon.0) 1329 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.877300+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: cluster 2024-02-18T01:24:44.877465+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.879679+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.881540+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.882154+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1893728271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.882970+0000 mon.c (mon.2) 252 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.887592+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.888438+0000 mon.a (mon.0) 1326 : audit [INF] from='client.48667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.905617+0000 mon.c (mon.2) 253 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.905899+0000 mon.a (mon.0) 1327 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.906847+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.907099+0000 mon.a (mon.0) 1328 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.907514+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:45 smithi105 bash[19734]: audit 2024-02-18T01:24:44.907773+0000 mon.a (mon.0) 1329 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.877300+0000 mon.a (mon.0) 1322 : audit [INF] from='client.? v1:172.21.15.105:0/2225464804' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi105-61649-27"}]': finished 2024-02-18T01:24:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: cluster 2024-02-18T01:24:44.877465+0000 mon.a (mon.0) 1323 : cluster [DBG] osdmap e159: 8 total, 8 up, 8 in 2024-02-18T01:24:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.879679+0000 mon.a (mon.0) 1324 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.881540+0000 mon.b (mon.1) 148 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.882154+0000 mon.b (mon.1) 149 : audit [INF] from='client.? v1:172.21.15.105:0/1893728271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.882970+0000 mon.c (mon.2) 252 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.887592+0000 mon.a (mon.0) 1325 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.888438+0000 mon.a (mon.0) 1326 : audit [INF] from='client.48667 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.905617+0000 mon.c (mon.2) 253 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.905899+0000 mon.a (mon.0) 1327 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.906847+0000 mon.c (mon.2) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.907099+0000 mon.a (mon.0) 1328 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.907514+0000 mon.c (mon.2) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:45 smithi155 bash[20805]: audit 2024-02-18T01:24:44.907773+0000 mon.a (mon.0) 1329 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:46] "GET /metrics HTTP/1.1" 200 194832 "" "Prometheus/2.33.4" 2024-02-18T01:24:47.333 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompl(68903 ms total) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (116 ms) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (116 ms total) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (7 ms) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (4 ms) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (11 ms total) 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-02-18T01:24:47.334 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (23 ms) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (4 ms) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (28 ms total) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (38 ms) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (7 ms) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (45 ms total) 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-02-18T01:24:47.335 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1038 ms) 2024-02-18T01:24:47.336 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1038 ms total) 2024-02-18T01:24:47.336 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: 2024-02-18T01:24:47.336 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-02-18T01:24:47.336 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (107318 ms total) 2024-02-18T01:24:47.336 INFO:tasks.workunit.client.0.smithi105.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-02-18T01:24:47.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: cluster 2024-02-18T01:24:45.656966+0000 mgr.y (mgr.24475) 321 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:47.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.888308+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:47.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.893731+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:47.652 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.893839+0000 mon.a (mon.0) 1331 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.894036+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.894293+0000 mon.a (mon.0) 1333 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: cluster 2024-02-18T01:24:45.894398+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.895019+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.897631+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:47.653 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[20805]: audit 2024-02-18T01:24:45.898070+0000 mon.a (mon.0) 1336 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: cluster 2024-02-18T01:24:45.656966+0000 mgr.y (mgr.24475) 321 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.888308+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.893731+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.893839+0000 mon.a (mon.0) 1331 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.894036+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.894293+0000 mon.a (mon.0) 1333 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: cluster 2024-02-18T01:24:45.894398+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.895019+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.897631+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[19734]: audit 2024-02-18T01:24:45.898070+0000 mon.a (mon.0) 1336 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: cluster 2024-02-18T01:24:45.656966+0000 mgr.y (mgr.24475) 321 : cluster [DBG] pgmap v316: 300 pgs: 64 unknown, 236 active+clean; 33 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 14 KiB/s wr, 9 op/s; 31 B/s, 0 objects/s recovering 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.888308+0000 mon.c (mon.2) 256 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.893731+0000 mon.a (mon.0) 1330 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.893839+0000 mon.a (mon.0) 1331 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.894036+0000 mon.a (mon.0) 1332 : audit [INF] from='client.48667 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi105-61657-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.894293+0000 mon.a (mon.0) 1333 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi105-61649-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: cluster 2024-02-18T01:24:45.894398+0000 mon.a (mon.0) 1334 : cluster [DBG] osdmap e160: 8 total, 8 up, 8 in 2024-02-18T01:24:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.895019+0000 mon.c (mon.2) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.897631+0000 mon.a (mon.0) 1335 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]: dispatch 2024-02-18T01:24:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:47 smithi105 bash[27669]: audit 2024-02-18T01:24:45.898070+0000 mon.a (mon.0) 1336 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:47.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:48.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:46.889430+0000 mon.c (mon.2) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:46.897740+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: cluster 2024-02-18T01:24:47.321558+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.334659+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.335139+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.649755+0000 mgr.y (mgr.24475) 322 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: cluster 2024-02-18T01:24:47.659316+0000 mgr.y (mgr.24475) 323 : cluster [DBG] pgmap v319: 267 pgs: 267 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.790750+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.891150+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.908338+0000 mon.a (mon.0) 1341 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.908481+0000 mon.a (mon.0) 1342 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.908606+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: cluster 2024-02-18T01:24:47.908676+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-18T01:24:48.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.910580+0000 mon.c (mon.2) 260 : audit [INF] from='client.? v1:172.21.15.105:0/3102593926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.912686+0000 mon.a (mon.0) 1345 : audit [INF] from='client.47930 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.914486+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[19734]: audit 2024-02-18T01:24:47.914982+0000 mon.a (mon.0) 1346 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:46.889430+0000 mon.c (mon.2) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:46.897740+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: cluster 2024-02-18T01:24:47.321558+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.334659+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.335139+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.649755+0000 mgr.y (mgr.24475) 322 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: cluster 2024-02-18T01:24:47.659316+0000 mgr.y (mgr.24475) 323 : cluster [DBG] pgmap v319: 267 pgs: 267 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:48.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.790750+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.891150+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.908338+0000 mon.a (mon.0) 1341 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.908481+0000 mon.a (mon.0) 1342 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.908606+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: cluster 2024-02-18T01:24:47.908676+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.910580+0000 mon.c (mon.2) 260 : audit [INF] from='client.? v1:172.21.15.105:0/3102593926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.912686+0000 mon.a (mon.0) 1345 : audit [INF] from='client.47930 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.914486+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:48.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:48 smithi105 bash[27669]: audit 2024-02-18T01:24:47.914982+0000 mon.a (mon.0) 1346 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:48.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:46.889430+0000 mon.c (mon.2) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:46.897740+0000 mon.a (mon.0) 1337 : audit [INF] from='client.? v1:172.21.15.105:0/1865835864' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi105-61790-36"}]': finished 2024-02-18T01:24:48.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: cluster 2024-02-18T01:24:47.321558+0000 mon.a (mon.0) 1338 : cluster [DBG] osdmap e161: 8 total, 8 up, 8 in 2024-02-18T01:24:48.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.334659+0000 mon.b (mon.1) 150 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.335139+0000 mon.a (mon.0) 1339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.649755+0000 mgr.y (mgr.24475) 322 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: cluster 2024-02-18T01:24:47.659316+0000 mgr.y (mgr.24475) 323 : cluster [DBG] pgmap v319: 267 pgs: 267 active+clean; 41 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.790750+0000 mon.a (mon.0) 1340 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.891150+0000 mon.c (mon.2) 259 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:48.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.908338+0000 mon.a (mon.0) 1341 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi105-61649-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.908481+0000 mon.a (mon.0) 1342 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.908606+0000 mon.a (mon.0) 1343 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi105-61689-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: cluster 2024-02-18T01:24:47.908676+0000 mon.a (mon.0) 1344 : cluster [DBG] osdmap e162: 8 total, 8 up, 8 in 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.910580+0000 mon.c (mon.2) 260 : audit [INF] from='client.? v1:172.21.15.105:0/3102593926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.912686+0000 mon.a (mon.0) 1345 : audit [INF] from='client.47930 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.914486+0000 mon.b (mon.1) 151 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:48.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:48 smithi155 bash[20805]: audit 2024-02-18T01:24:47.914982+0000 mon.a (mon.0) 1346 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.806740+0000 mon.a (mon.0) 1347 : audit [INF] from='client.47930 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.807155+0000 mon.a (mon.0) 1348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: cluster 2024-02-18T01:24:48.807646+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-18T01:24:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.809492+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.809871+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.813011+0000 mon.a (mon.0) 1350 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.813574+0000 mon.a (mon.0) 1351 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[27669]: audit 2024-02-18T01:24:48.892750+0000 mon.c (mon.2) 261 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.806740+0000 mon.a (mon.0) 1347 : audit [INF] from='client.47930 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.807155+0000 mon.a (mon.0) 1348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: cluster 2024-02-18T01:24:48.807646+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.809492+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.809871+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.813011+0000 mon.a (mon.0) 1350 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.813574+0000 mon.a (mon.0) 1351 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:49 smithi105 bash[19734]: audit 2024-02-18T01:24:48.892750+0000 mon.c (mon.2) 261 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.806740+0000 mon.a (mon.0) 1347 : audit [INF] from='client.47930 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi105-61657-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.807155+0000 mon.a (mon.0) 1348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: cluster 2024-02-18T01:24:48.807646+0000 mon.a (mon.0) 1349 : cluster [DBG] osdmap e163: 8 total, 8 up, 8 in 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.809492+0000 mon.b (mon.1) 152 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.809871+0000 mon.b (mon.1) 153 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.813011+0000 mon.a (mon.0) 1350 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]: dispatch 2024-02-18T01:24:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.813574+0000 mon.a (mon.0) 1351 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:50.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:49 smithi155 bash[20805]: audit 2024-02-18T01:24:48.892750+0000 mon.c (mon.2) 261 : audit [DBG] from='client.? v1:172.21.15.105:0/2745248100' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-02-18T01:24:50.837 INFO:tasks.workunit.client.0.smithi105.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4465 ms) 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3574 ms) 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4033 ms) 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12072 ms total) 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (110778 ms total) 2024-02-18T01:24:50.838 INFO:tasks.workunit.client.0.smithi105.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-02-18T01:24:50.839 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:50.839 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61705 2024-02-18T01:24:50.839 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61705 2024-02-18T01:24:50.839 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:50.839 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61700 2024-02-18T01:24:50.840 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61700 2024-02-18T01:24:50.840 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:50.840 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61684 2024-02-18T01:24:50.840 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61684 2024-02-18T01:24:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: cluster 2024-02-18T01:24:49.660343+0000 mgr.y (mgr.24475) 324 : cluster [DBG] pgmap v322: 299 pgs: 40 unknown, 259 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: cluster 2024-02-18T01:24:49.800293+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: cluster 2024-02-18T01:24:49.801366+0000 mon.a (mon.0) 1353 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.809234+0000 mon.a (mon.0) 1354 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]': finished 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.809501+0000 mon.a (mon.0) 1355 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: cluster 2024-02-18T01:24:49.809866+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.812587+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.815308+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.821380+0000 mon.a (mon.0) 1357 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.821642+0000 mon.a (mon.0) 1358 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:49.845119+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:50.272624+0000 mon.a (mon.0) 1359 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:50.273651+0000 mon.c (mon.2) 264 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:50.274131+0000 mon.a (mon.0) 1360 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:50.275101+0000 mon.c (mon.2) 265 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[19734]: audit 2024-02-18T01:24:50.275641+0000 mon.a (mon.0) 1361 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: cluster 2024-02-18T01:24:49.660343+0000 mgr.y (mgr.24475) 324 : cluster [DBG] pgmap v322: 299 pgs: 40 unknown, 259 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: cluster 2024-02-18T01:24:49.800293+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: cluster 2024-02-18T01:24:49.801366+0000 mon.a (mon.0) 1353 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.809234+0000 mon.a (mon.0) 1354 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]': finished 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.809501+0000 mon.a (mon.0) 1355 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: cluster 2024-02-18T01:24:49.809866+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.812587+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.815308+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.821380+0000 mon.a (mon.0) 1357 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.821642+0000 mon.a (mon.0) 1358 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:49.845119+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:50.272624+0000 mon.a (mon.0) 1359 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:50.273651+0000 mon.c (mon.2) 264 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:50.274131+0000 mon.a (mon.0) 1360 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:50.275101+0000 mon.c (mon.2) 265 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:51.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:50 smithi105 bash[27669]: audit 2024-02-18T01:24:50.275641+0000 mon.a (mon.0) 1361 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: cluster 2024-02-18T01:24:49.660343+0000 mgr.y (mgr.24475) 324 : cluster [DBG] pgmap v322: 299 pgs: 40 unknown, 259 active+clean; 8.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: cluster 2024-02-18T01:24:49.800293+0000 mon.a (mon.0) 1352 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: cluster 2024-02-18T01:24:49.801366+0000 mon.a (mon.0) 1353 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.809234+0000 mon.a (mon.0) 1354 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-15", "mode": "writeback"}]': finished 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.809501+0000 mon.a (mon.0) 1355 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: cluster 2024-02-18T01:24:49.809866+0000 mon.a (mon.0) 1356 : cluster [DBG] osdmap e164: 8 total, 8 up, 8 in 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.812587+0000 mon.b (mon.1) 154 : audit [INF] from='client.? v1:172.21.15.105:0/3792584376' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.815308+0000 mon.c (mon.2) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.821380+0000 mon.a (mon.0) 1357 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.821642+0000 mon.a (mon.0) 1358 : audit [INF] from='client.48622 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:49.845119+0000 mon.c (mon.2) 263 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:50.272624+0000 mon.a (mon.0) 1359 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:50.273651+0000 mon.c (mon.2) 264 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:50.274131+0000 mon.a (mon.0) 1360 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:50.275101+0000 mon.c (mon.2) 265 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:50 smithi155 bash[20805]: audit 2024-02-18T01:24:50.275641+0000 mon.a (mon.0) 1361 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:24:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:24:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.829753+0000 mon.a (mon.0) 1362 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.829869+0000 mon.a (mon.0) 1363 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.830022+0000 mon.a (mon.0) 1364 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: cluster 2024-02-18T01:24:50.830125+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.831423+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.831938+0000 mon.c (mon.2) 267 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.833048+0000 mon.c (mon.2) 268 : audit [INF] from='client.? v1:172.21.15.105:0/2350983441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.838433+0000 mon.a (mon.0) 1366 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.838966+0000 mon.a (mon.0) 1367 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.839890+0000 mon.a (mon.0) 1368 : audit [INF] from='client.47936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.867175+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[19734]: audit 2024-02-18T01:24:50.867633+0000 mon.a (mon.0) 1369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.829753+0000 mon.a (mon.0) 1362 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.829869+0000 mon.a (mon.0) 1363 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.830022+0000 mon.a (mon.0) 1364 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: cluster 2024-02-18T01:24:50.830125+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.831423+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.831938+0000 mon.c (mon.2) 267 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.833048+0000 mon.c (mon.2) 268 : audit [INF] from='client.? v1:172.21.15.105:0/2350983441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.838433+0000 mon.a (mon.0) 1366 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.838966+0000 mon.a (mon.0) 1367 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.839890+0000 mon.a (mon.0) 1368 : audit [INF] from='client.47936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.867175+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:52 smithi105 bash[27669]: audit 2024-02-18T01:24:50.867633+0000 mon.a (mon.0) 1369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.829753+0000 mon.a (mon.0) 1362 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.829869+0000 mon.a (mon.0) 1363 : audit [INF] from='client.48622 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi105-61914-20"}]': finished 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.830022+0000 mon.a (mon.0) 1364 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi105-61689-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: cluster 2024-02-18T01:24:50.830125+0000 mon.a (mon.0) 1365 : cluster [DBG] osdmap e165: 8 total, 8 up, 8 in 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.831423+0000 mon.c (mon.2) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2209257107' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.831938+0000 mon.c (mon.2) 267 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.833048+0000 mon.c (mon.2) 268 : audit [INF] from='client.? v1:172.21.15.105:0/2350983441' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.838433+0000 mon.a (mon.0) 1366 : audit [INF] from='client.47924 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.838966+0000 mon.a (mon.0) 1367 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.839890+0000 mon.a (mon.0) 1368 : audit [INF] from='client.47936 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.867175+0000 mon.b (mon.1) 155 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:52.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:52 smithi155 bash[20805]: audit 2024-02-18T01:24:50.867633+0000 mon.a (mon.0) 1369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: cluster 2024-02-18T01:24:51.661293+0000 mgr.y (mgr.24475) 325 : cluster [DBG] pgmap v325: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:51.837029+0000 mon.a (mon.0) 1370 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:51.859956+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.252623+0000 mon.a (mon.0) 1371 : audit [INF] from='client.47936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.253032+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: cluster 2024-02-18T01:24:52.253296+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.254452+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.255228+0000 mon.a (mon.0) 1374 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.255815+0000 mon.c (mon.2) 270 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.260347+0000 mon.a (mon.0) 1375 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.260923+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.265801+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.266018+0000 mon.a (mon.0) 1377 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: cluster 2024-02-18T01:24:52.824974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.836312+0000 mon.a (mon.0) 1379 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:53.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: cluster 2024-02-18T01:24:51.661293+0000 mgr.y (mgr.24475) 325 : cluster [DBG] pgmap v325: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:51.837029+0000 mon.a (mon.0) 1370 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:51.859956+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.252623+0000 mon.a (mon.0) 1371 : audit [INF] from='client.47936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.253032+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: cluster 2024-02-18T01:24:52.253296+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.254452+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.255228+0000 mon.a (mon.0) 1374 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.255815+0000 mon.c (mon.2) 270 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.260347+0000 mon.a (mon.0) 1375 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.260923+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.265801+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.266018+0000 mon.a (mon.0) 1377 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: cluster 2024-02-18T01:24:52.824974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:53.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.836312+0000 mon.a (mon.0) 1379 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.836461+0000 mon.a (mon.0) 1380 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.836571+0000 mon.a (mon.0) 1381 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: cluster 2024-02-18T01:24:52.836703+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.838301+0000 mon.c (mon.2) 272 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[19734]: audit 2024-02-18T01:24:52.840548+0000 mon.a (mon.0) 1383 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.836461+0000 mon.a (mon.0) 1380 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.836571+0000 mon.a (mon.0) 1381 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:53.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: cluster 2024-02-18T01:24:52.836703+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.838301+0000 mon.c (mon.2) 272 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:53 smithi105 bash[27669]: audit 2024-02-18T01:24:52.840548+0000 mon.a (mon.0) 1383 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: cluster 2024-02-18T01:24:51.661293+0000 mgr.y (mgr.24475) 325 : cluster [DBG] pgmap v325: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:51.837029+0000 mon.a (mon.0) 1370 : audit [INF] from='client.47924 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi105-61649-28"}]': finished 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:51.859956+0000 mon.c (mon.2) 269 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.252623+0000 mon.a (mon.0) 1371 : audit [INF] from='client.47936 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi105-61657-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.253032+0000 mon.a (mon.0) 1372 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: cluster 2024-02-18T01:24:52.253296+0000 mon.a (mon.0) 1373 : cluster [DBG] osdmap e166: 8 total, 8 up, 8 in 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.254452+0000 mon.b (mon.1) 156 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.255228+0000 mon.a (mon.0) 1374 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.255815+0000 mon.c (mon.2) 270 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.260347+0000 mon.a (mon.0) 1375 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.260923+0000 mon.c (mon.2) 271 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.265801+0000 mon.a (mon.0) 1376 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.266018+0000 mon.a (mon.0) 1377 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: cluster 2024-02-18T01:24:52.824974+0000 mon.a (mon.0) 1378 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.836312+0000 mon.a (mon.0) 1379 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi105-61689-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.836461+0000 mon.a (mon.0) 1380 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-15"}]': finished 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.836571+0000 mon.a (mon.0) 1381 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi105-61649-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:24:53.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: cluster 2024-02-18T01:24:52.836703+0000 mon.a (mon.0) 1382 : cluster [DBG] osdmap e167: 8 total, 8 up, 8 in 2024-02-18T01:24:53.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.838301+0000 mon.c (mon.2) 272 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:53 smithi155 bash[20805]: audit 2024-02-18T01:24:52.840548+0000 mon.a (mon.0) 1383 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:53.914 INFO:tasks.workunit.client.0.smithi105.stdout:d=8 2024-02-18T01:24:53.914 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:b29083e3:::5:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:b29083e3:::5:head expected=16:b29083e3:::5:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:b29083e3:::5:head -> 5 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=5 expected=5 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:89d3ae78:::11:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:89d3ae78:::11:head expected=16:89d3ae78:::11:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:89d3ae78:::11:head -> 11 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=11 expected=11 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:566253c9:::13:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:566253c9:::13:head expected=16:566253c9:::13:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:566253c9:::13:head -> 13 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=13 expected=13 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:02547ec2:::1:head 2024-02-18T01:24:53.915 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:02547ec2:::1:head expected=16:02547ec2:::1:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:02547ec2:::1:head -> 1 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=1 expected=1 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:863748b0:::15:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:863748b0:::15:head expected=16:863748b0:::15:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:863748b0:::15:head -> 15 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=15 expected=15 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:62a1935d:::14:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:62a1935d:::14:head expected=16:62a1935d:::14:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:62a1935d:::14:head -> 14 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=14 expected=14 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:5c6b0b28:::7:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:5c6b0b28:::7:head expected=16:5c6b0b28:::7:head 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:5c6b0b28:::7:head -> 7 2024-02-18T01:24:53.916 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=7 expected=7 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : seek to 16:52ea6a34:::10:head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : cursor()=16:52ea6a34:::10:head expected=16:52ea6a34:::10:head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: > 16:52ea6a34:::10:head -> 10 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: : entry=10 expected=10 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (261 ms) 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (86688 ms) 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: split 0/5 -> MIN 16:33333333::::head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: split 1/5 -> 16:33333333::::head 16:66666666::::head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: split 2/5 -> 16:66666666::::head 16:99999999::::head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: split 3/5 -> 16:99999999::::head 16:cccccccc::::head 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: split 4/5 -> 16:cccccccc::::head MAX 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (10023 ms) 2024-02-18T01:24:53.917 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 7 tests from LibRadosList (97534 ms total) 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1041 ms) 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo1,foo2,foo3 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo1 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo2 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo3 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo1,foo4,foo5 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo4 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo5 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo1 2024-02-18T01:24:53.918 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset foo6,foo7 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo7 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: foo6 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo4 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo5 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns2:foo7 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns2:foo6 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: ns1:foo1 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo1 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo2 2024-02-18T01:24:53.919 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: :foo3 2024-02-18T01:24:53.920 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (33 ms) 2024-02-18T01:24:55.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[19734]: cluster 2024-02-18T01:24:53.662142+0000 mgr.y (mgr.24475) 326 : cluster [DBG] pgmap v328: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:55.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[19734]: cluster 2024-02-18T01:24:54.267865+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-18T01:24:55.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[19734]: audit 2024-02-18T01:24:54.269971+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[27669]: cluster 2024-02-18T01:24:53.662142+0000 mgr.y (mgr.24475) 326 : cluster [DBG] pgmap v328: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[27669]: cluster 2024-02-18T01:24:54.267865+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-18T01:24:55.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:54 smithi105 bash[27669]: audit 2024-02-18T01:24:54.269971+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:54 smithi155 bash[20805]: cluster 2024-02-18T01:24:53.662142+0000 mgr.y (mgr.24475) 326 : cluster [DBG] pgmap v328: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:54 smithi155 bash[20805]: cluster 2024-02-18T01:24:54.267865+0000 mon.a (mon.0) 1384 : cluster [DBG] osdmap e168: 8 total, 8 up, 8 in 2024-02-18T01:24:55.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:54 smithi155 bash[20805]: audit 2024-02-18T01:24:54.269971+0000 mon.a (mon.0) 1385 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosListEC.Lis api_aio_pp: Running main() from gmock_main.cc 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [==========] Running 56 tests from 4 test suites. 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (13017 ms) 2024-02-18T01:24:55.858 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (22741 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6605 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3112 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3228 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (3120 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3101 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3020 ms) 2024-02-18T01:24:55.859 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3914 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2986 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3033 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2978 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3015 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3043 ms) 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-02-18T01:24:55.860 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3467 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3008 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2605 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2867 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3013 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3437 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (2600 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3012 ms) 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-02-18T01:24:55.861 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3016 ms) 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (3437 ms) 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2492 ms) 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3023 ms) 2024-02-18T01:24:55.862 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-02-18T01:24:56.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.847132+0000 mon.a (mon.0) 1386 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:56.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.847483+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:56.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: cluster 2024-02-18T01:24:54.847860+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-18T01:24:56.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.849035+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.851326+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.852662+0000 mon.a (mon.0) 1389 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[27669]: audit 2024-02-18T01:24:54.852901+0000 mon.a (mon.0) 1390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.847132+0000 mon.a (mon.0) 1386 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.847483+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: cluster 2024-02-18T01:24:54.847860+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.849035+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.851326+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.852662+0000 mon.a (mon.0) 1389 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:55 smithi105 bash[19734]: audit 2024-02-18T01:24:54.852901+0000 mon.a (mon.0) 1390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.847132+0000 mon.a (mon.0) 1386 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi105-61649-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.847483+0000 mon.a (mon.0) 1387 : audit [INF] from='client.? v1:172.21.15.105:0/1833093079' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi105-61657-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: cluster 2024-02-18T01:24:54.847860+0000 mon.a (mon.0) 1388 : cluster [DBG] osdmap e169: 8 total, 8 up, 8 in 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.849035+0000 mon.c (mon.2) 273 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.851326+0000 mon.b (mon.1) 157 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.852662+0000 mon.a (mon.0) 1389 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:55 smithi155 bash[20805]: audit 2024-02-18T01:24:54.852901+0000 mon.a (mon.0) 1390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: cluster 2024-02-18T01:24:55.663083+0000 mgr.y (mgr.24475) 327 : cluster [DBG] pgmap v331: 268 pgs: 72 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:55.851346+0000 mon.a (mon.0) 1391 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:55.851583+0000 mon.a (mon.0) 1392 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: cluster 2024-02-18T01:24:55.852354+0000 mon.a (mon.0) 1393 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:55.858371+0000 mon.c (mon.2) 274 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:55.875378+0000 mon.a (mon.0) 1394 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:55.895025+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[27669]: audit 2024-02-18T01:24:56.325373+0000 mon.a (mon.0) 1395 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: cluster 2024-02-18T01:24:55.663083+0000 mgr.y (mgr.24475) 327 : cluster [DBG] pgmap v331: 268 pgs: 72 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:55.851346+0000 mon.a (mon.0) 1391 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:55.851583+0000 mon.a (mon.0) 1392 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: cluster 2024-02-18T01:24:55.852354+0000 mon.a (mon.0) 1393 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:55.858371+0000 mon.c (mon.2) 274 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:55.875378+0000 mon.a (mon.0) 1394 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:55.895025+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:56.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[19734]: audit 2024-02-18T01:24:56.325373+0000 mon.a (mon.0) 1395 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:56.993 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:24:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:24:56] "GET /metrics HTTP/1.1" 200 187847 "" "Prometheus/2.33.4" 2024-02-18T01:24:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: cluster 2024-02-18T01:24:55.663083+0000 mgr.y (mgr.24475) 327 : cluster [DBG] pgmap v331: 268 pgs: 72 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:24:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:55.851346+0000 mon.a (mon.0) 1391 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:55.851583+0000 mon.a (mon.0) 1392 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: cluster 2024-02-18T01:24:55.852354+0000 mon.a (mon.0) 1393 : cluster [DBG] osdmap e170: 8 total, 8 up, 8 in 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:55.858371+0000 mon.c (mon.2) 274 : audit [INF] from='client.? v1:172.21.15.105:0/118421158' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:55.875378+0000 mon.a (mon.0) 1394 : audit [INF] from='client.47942 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]: dispatch 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:55.895025+0000 mon.b (mon.1) 158 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:57.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:56 smithi155 bash[20805]: audit 2024-02-18T01:24:56.325373+0000 mon.a (mon.0) 1395 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:24:57.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:24:57.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.882355+0000 mon.a (mon.0) 1396 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:57.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.882484+0000 mon.a (mon.0) 1397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:57.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: cluster 2024-02-18T01:24:56.882535+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-18T01:24:57.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.887044+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:57.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.890482+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:57.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.891614+0000 mon.a (mon.0) 1400 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:57.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.898022+0000 mon.c (mon.2) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:57.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:57 smithi155 bash[20805]: audit 2024-02-18T01:24:56.898960+0000 mon.a (mon.0) 1401 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.882355+0000 mon.a (mon.0) 1396 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.882484+0000 mon.a (mon.0) 1397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: cluster 2024-02-18T01:24:56.882535+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.887044+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.890482+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.891614+0000 mon.a (mon.0) 1400 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.898022+0000 mon.c (mon.2) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[19734]: audit 2024-02-18T01:24:56.898960+0000 mon.a (mon.0) 1401 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.882355+0000 mon.a (mon.0) 1396 : audit [INF] from='client.47942 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi105-61689-2"}]': finished 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.882484+0000 mon.a (mon.0) 1397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: cluster 2024-02-18T01:24:56.882535+0000 mon.a (mon.0) 1398 : cluster [DBG] osdmap e171: 8 total, 8 up, 8 in 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.887044+0000 mon.a (mon.0) 1399 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.890482+0000 mon.b (mon.1) 159 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.891614+0000 mon.a (mon.0) 1400 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:24:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.898022+0000 mon.c (mon.2) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:58.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:57 smithi105 bash[27669]: audit 2024-02-18T01:24:56.898960+0000 mon.a (mon.0) 1401 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.660934+0000 mgr.y (mgr.24475) 328 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:59.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: cluster 2024-02-18T01:24:57.665310+0000 mgr.y (mgr.24475) 329 : cluster [DBG] pgmap v334: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:59.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.889027+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.889270+0000 mon.a (mon.0) 1403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.889425+0000 mon.a (mon.0) 1404 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: cluster 2024-02-18T01:24:57.889560+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.891229+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.891426+0000 mon.c (mon.2) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.892630+0000 mon.a (mon.0) 1406 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.893119+0000 mon.a (mon.0) 1407 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.893716+0000 mon.c (mon.2) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:57.908221+0000 mon.a (mon.0) 1408 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: cluster 2024-02-18T01:24:58.802378+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:59.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.810680+0000 mon.a (mon.0) 1410 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.810984+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]': finished 2024-02-18T01:24:59.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.660934+0000 mgr.y (mgr.24475) 328 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:59.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: cluster 2024-02-18T01:24:57.665310+0000 mgr.y (mgr.24475) 329 : cluster [DBG] pgmap v334: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:59.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.889027+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.889270+0000 mon.a (mon.0) 1403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.889425+0000 mon.a (mon.0) 1404 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: cluster 2024-02-18T01:24:57.889560+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.891229+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.891426+0000 mon.c (mon.2) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.892630+0000 mon.a (mon.0) 1406 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.893119+0000 mon.a (mon.0) 1407 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.893716+0000 mon.c (mon.2) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:57.908221+0000 mon.a (mon.0) 1408 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: cluster 2024-02-18T01:24:58.802378+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.810680+0000 mon.a (mon.0) 1410 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.810984+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]': finished 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.811193+0000 mon.a (mon.0) 1412 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: cluster 2024-02-18T01:24:58.811347+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-18T01:24:59.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.850773+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.851592+0000 mon.a (mon.0) 1414 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.857605+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.858081+0000 mon.a (mon.0) 1415 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.859105+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.859965+0000 mon.a (mon.0) 1416 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.747 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[27669]: audit 2024-02-18T01:24:58.860609+0000 mon.c (mon.2) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.811193+0000 mon.a (mon.0) 1412 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: cluster 2024-02-18T01:24:58.811347+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.850773+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.851592+0000 mon.a (mon.0) 1414 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.857605+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.748 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.858081+0000 mon.a (mon.0) 1415 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.859105+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.859965+0000 mon.a (mon.0) 1416 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.749 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:24:59 smithi105 bash[19734]: audit 2024-02-18T01:24:58.860609+0000 mon.c (mon.2) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:24:59.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.660934+0000 mgr.y (mgr.24475) 328 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:24:59.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: cluster 2024-02-18T01:24:57.665310+0000 mgr.y (mgr.24475) 329 : cluster [DBG] pgmap v334: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:24:59.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.889027+0000 mon.a (mon.0) 1402 : audit [INF] from='client.? v1:172.21.15.105:0/601215663' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi105-61657-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.750 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.889270+0000 mon.a (mon.0) 1403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.889425+0000 mon.a (mon.0) 1404 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: cluster 2024-02-18T01:24:57.889560+0000 mon.a (mon.0) 1405 : cluster [DBG] osdmap e172: 8 total, 8 up, 8 in 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.891229+0000 mon.b (mon.1) 160 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.891426+0000 mon.c (mon.2) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2308860599' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.892630+0000 mon.a (mon.0) 1406 : audit [INF] from='client.47948 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]: dispatch 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.893119+0000 mon.a (mon.0) 1407 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]: dispatch 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.893716+0000 mon.c (mon.2) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.751 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:57.908221+0000 mon.a (mon.0) 1408 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: cluster 2024-02-18T01:24:58.802378+0000 mon.a (mon.0) 1409 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.810680+0000 mon.a (mon.0) 1410 : audit [INF] from='client.47948 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi105-61649-29"}]': finished 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.810984+0000 mon.a (mon.0) 1411 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-17", "mode": "writeback"}]': finished 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.811193+0000 mon.a (mon.0) 1412 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61689-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: cluster 2024-02-18T01:24:58.811347+0000 mon.a (mon.0) 1413 : cluster [DBG] osdmap e173: 8 total, 8 up, 8 in 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.850773+0000 mon.b (mon.1) 161 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.851592+0000 mon.a (mon.0) 1414 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.857605+0000 mon.b (mon.1) 162 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.752 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.858081+0000 mon.a (mon.0) 1415 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:24:59.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.859105+0000 mon.b (mon.1) 163 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.859965+0000 mon.a (mon.0) 1416 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:24:59.753 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:24:59 smithi155 bash[20805]: audit 2024-02-18T01:24:58.860609+0000 mon.c (mon.2) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2990977490' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:24:59.826 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAitObjectsStart 2024-02-18T01:24:59.826 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 1 0 2024-02-18T01:24:59.826 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 10 0 2024-02-18T01:24:59.826 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 13 0 2024-02-18T01:24:59.827 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 7 0 2024-02-18T01:24:59.828 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 14 0 2024-02-18T01:24:59.828 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 0 0 2024-02-18T01:24:59.828 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 15 0 2024-02-18T01:24:59.828 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 11 0 2024-02-18T01:24:59.828 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 5 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 8 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 6 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 3 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 4 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 12 0 2024-02-18T01:24:59.829 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 9 0 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 2 0 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (50 ms) 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 3 tests from LibRadosListEC (1124 ms total) 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 2024-02-18T01:24:59.830 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (2931 ms) 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] 1 test from LibRadosListNP (2931 ms total) 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [----------] Global test environment tear-down 2024-02-18T01:24:59.831 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [==========] 11 tests from 3 test suites ran. (119847 ms total) 2024-02-18T01:24:59.832 INFO:tasks.workunit.client.0.smithi105.stdout: api_list: [ PASSED ] 11 tests. 2024-02-18T01:24:59.833 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:59.833 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61955 2024-02-18T01:24:59.833 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61955 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61944 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61944 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62145 2024-02-18T01:24:59.834 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62145 2024-02-18T01:24:59.835 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:24:59.835 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61652 2024-02-18T01:24:59.835 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61652 2024-02-18T01:25:00.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:58.908348+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.289227+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.289882+0000 mon.a (mon.0) 1418 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: cluster 2024-02-18T01:24:59.666199+0000 mgr.y (mgr.24475) 330 : cluster [DBG] pgmap v337: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.814781+0000 mon.a (mon.0) 1419 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.815249+0000 mon.a (mon.0) 1420 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.815352+0000 mon.a (mon.0) 1421 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: cluster 2024-02-18T01:24:59.815816+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.816170+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.817960+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.818840+0000 mon.a (mon.0) 1423 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.820763+0000 mon.a (mon.0) 1424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.822973+0000 mon.c (mon.2) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2552480841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[19734]: audit 2024-02-18T01:24:59.823744+0000 mon.a (mon.0) 1425 : audit [INF] from='client.47975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:00.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:58.908348+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.289227+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.289882+0000 mon.a (mon.0) 1418 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: cluster 2024-02-18T01:24:59.666199+0000 mgr.y (mgr.24475) 330 : cluster [DBG] pgmap v337: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.814781+0000 mon.a (mon.0) 1419 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.815249+0000 mon.a (mon.0) 1420 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.815352+0000 mon.a (mon.0) 1421 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: cluster 2024-02-18T01:24:59.815816+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.816170+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.817960+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.818840+0000 mon.a (mon.0) 1423 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.820763+0000 mon.a (mon.0) 1424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.822973+0000 mon.c (mon.2) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2552480841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:00.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:00 smithi105 bash[27669]: audit 2024-02-18T01:24:59.823744+0000 mon.a (mon.0) 1425 : audit [INF] from='client.47975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:00.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:58.908348+0000 mon.b (mon.1) 164 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.289227+0000 mon.a (mon.0) 1417 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:00.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.289882+0000 mon.a (mon.0) 1418 : audit [INF] from='client.47963 ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-02-18T01:25:00.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: cluster 2024-02-18T01:24:59.666199+0000 mgr.y (mgr.24475) 330 : cluster [DBG] pgmap v337: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.814781+0000 mon.a (mon.0) 1419 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi105-61649-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.815249+0000 mon.a (mon.0) 1420 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.815352+0000 mon.a (mon.0) 1421 : audit [INF] from='client.47963 ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi105-61689-3","pool2":"test-rados-api-smithi105-61689-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: cluster 2024-02-18T01:24:59.815816+0000 mon.a (mon.0) 1422 : cluster [DBG] osdmap e174: 8 total, 8 up, 8 in 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.816170+0000 mon.b (mon.1) 165 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.817960+0000 mon.b (mon.1) 166 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.818840+0000 mon.a (mon.0) 1423 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.820763+0000 mon.a (mon.0) 1424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]: dispatch 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.822973+0000 mon.c (mon.2) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2552480841' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:00.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:00 smithi155 bash[20805]: audit 2024-02-18T01:24:59.823744+0000 mon.a (mon.0) 1425 : audit [INF] from='client.47975 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:01.661 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:01 smithi155 bash[20805]: cluster 2024-02-18T01:25:00.809727+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:01.661 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:01 smithi155 bash[20805]: audit 2024-02-18T01:25:01.245937+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:25:01.661 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:01 smithi155 bash[20805]: audit 2024-02-18T01:25:01.246309+0000 mon.a (mon.0) 1428 : audit [INF] from='client.47975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:01.662 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:01 smithi155 bash[20805]: cluster 2024-02-18T01:25:01.246916+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-18T01:25:01.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[19734]: cluster 2024-02-18T01:25:00.809727+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:01.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[19734]: audit 2024-02-18T01:25:01.245937+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:25:01.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[19734]: audit 2024-02-18T01:25:01.246309+0000 mon.a (mon.0) 1428 : audit [INF] from='client.47975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:01.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[19734]: cluster 2024-02-18T01:25:01.246916+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-18T01:25:01.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[27669]: cluster 2024-02-18T01:25:00.809727+0000 mon.a (mon.0) 1426 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:01.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[27669]: audit 2024-02-18T01:25:01.245937+0000 mon.a (mon.0) 1427 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-17"}]': finished 2024-02-18T01:25:01.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[27669]: audit 2024-02-18T01:25:01.246309+0000 mon.a (mon.0) 1428 : audit [INF] from='client.47975 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi105-61657-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:01.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:01 smithi105 bash[27669]: cluster 2024-02-18T01:25:01.246916+0000 mon.a (mon.0) 1429 : cluster [DBG] osdmap e175: 8 total, 8 up, 8 in 2024-02-18T01:25:01.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[19734]: cluster 2024-02-18T01:25:01.667090+0000 mgr.y (mgr.24475) 331 : cluster [DBG] pgmap v340: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[19734]: audit 2024-02-18T01:25:01.821898+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[19734]: cluster 2024-02-18T01:25:01.822263+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-18T01:25:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[27669]: cluster 2024-02-18T01:25:01.667090+0000 mgr.y (mgr.24475) 331 : cluster [DBG] pgmap v340: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[27669]: audit 2024-02-18T01:25:01.821898+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:02 smithi105 bash[27669]: cluster 2024-02-18T01:25:01.822263+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-18T01:25:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:02 smithi155 bash[20805]: cluster 2024-02-18T01:25:01.667090+0000 mgr.y (mgr.24475) 331 : cluster [DBG] pgmap v340: 260 pgs: 32 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:02 smithi155 bash[20805]: audit 2024-02-18T01:25:01.821898+0000 mon.a (mon.0) 1430 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi105-61649-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:02 smithi155 bash[20805]: cluster 2024-02-18T01:25:01.822263+0000 mon.a (mon.0) 1431 : cluster [DBG] osdmap e176: 8 total, 8 up, 8 in 2024-02-18T01:25:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: cluster 2024-02-18T01:25:02.827878+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-18T01:25:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:02.837196+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:02.837388+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.105:0/3500883711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:02.838054+0000 mon.a (mon.0) 1433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:02.838490+0000 mon.a (mon.0) 1434 : audit [INF] from='client.48730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:03.829322+0000 mon.a (mon.0) 1435 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: audit 2024-02-18T01:25:03.829435+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[19734]: cluster 2024-02-18T01:25:03.829616+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: cluster 2024-02-18T01:25:02.827878+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:02.837196+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:02.837388+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.105:0/3500883711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:02.838054+0000 mon.a (mon.0) 1433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:02.838490+0000 mon.a (mon.0) 1434 : audit [INF] from='client.48730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:03.829322+0000 mon.a (mon.0) 1435 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: audit 2024-02-18T01:25:03.829435+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:03 smithi105 bash[27669]: cluster 2024-02-18T01:25:03.829616+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: cluster 2024-02-18T01:25:02.827878+0000 mon.a (mon.0) 1432 : cluster [DBG] osdmap e177: 8 total, 8 up, 8 in 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:02.837196+0000 mon.b (mon.1) 167 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:02.837388+0000 mon.b (mon.1) 168 : audit [INF] from='client.? v1:172.21.15.105:0/3500883711' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:02.838054+0000 mon.a (mon.0) 1433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:02.838490+0000 mon.a (mon.0) 1434 : audit [INF] from='client.48730 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:03.829322+0000 mon.a (mon.0) 1435 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: audit 2024-02-18T01:25:03.829435+0000 mon.a (mon.0) 1436 : audit [INF] from='client.48730 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi105-61657-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:04.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:03 smithi155 bash[20805]: cluster 2024-02-18T01:25:03.829616+0000 mon.a (mon.0) 1437 : cluster [DBG] osdmap e178: 8 total, 8 up, 8 in 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: cluster 2024-02-18T01:25:03.668067+0000 mgr.y (mgr.24475) 332 : cluster [DBG] pgmap v343: 268 pgs: 64 unknown, 8 creating+peering, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:03.833473+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:03.835578+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:03.836551+0000 mon.a (mon.0) 1438 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:03.850522+0000 mon.a (mon.0) 1439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:04.835103+0000 mon.a (mon.0) 1440 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: audit 2024-02-18T01:25:04.835321+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[27669]: cluster 2024-02-18T01:25:04.835453+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: cluster 2024-02-18T01:25:03.668067+0000 mgr.y (mgr.24475) 332 : cluster [DBG] pgmap v343: 268 pgs: 64 unknown, 8 creating+peering, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:03.833473+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:03.835578+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:03.836551+0000 mon.a (mon.0) 1438 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:03.850522+0000 mon.a (mon.0) 1439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:04.835103+0000 mon.a (mon.0) 1440 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: audit 2024-02-18T01:25:04.835321+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:04 smithi105 bash[19734]: cluster 2024-02-18T01:25:04.835453+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: cluster 2024-02-18T01:25:03.668067+0000 mgr.y (mgr.24475) 332 : cluster [DBG] pgmap v343: 268 pgs: 64 unknown, 8 creating+peering, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:03.833473+0000 mon.b (mon.1) 169 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:03.835578+0000 mon.b (mon.1) 170 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:03.836551+0000 mon.a (mon.0) 1438 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:03.850522+0000 mon.a (mon.0) 1439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:04.835103+0000 mon.a (mon.0) 1440 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: audit 2024-02-18T01:25:04.835321+0000 mon.a (mon.0) 1441 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:05.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:04 smithi155 bash[20805]: cluster 2024-02-18T01:25:04.835453+0000 mon.a (mon.0) 1442 : cluster [DBG] osdmap e179: 8 total, 8 up, 8 in 2024-02-18T01:25:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:04.837289+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:04.837944+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:04.839744+0000 mon.a (mon.0) 1443 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:04.840406+0000 mon.a (mon.0) 1444 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:05.840595+0000 mon.a (mon.0) 1445 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: audit 2024-02-18T01:25:05.840737+0000 mon.a (mon.0) 1446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[27669]: cluster 2024-02-18T01:25:05.840871+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:04.837289+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:04.837944+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:04.839744+0000 mon.a (mon.0) 1443 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:04.840406+0000 mon.a (mon.0) 1444 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:05.840595+0000 mon.a (mon.0) 1445 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: audit 2024-02-18T01:25:05.840737+0000 mon.a (mon.0) 1446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:05 smithi105 bash[19734]: cluster 2024-02-18T01:25:05.840871+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:04.837289+0000 mon.b (mon.1) 171 : audit [INF] from='client.? v1:172.21.15.105:0/1755867464' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:04.837944+0000 mon.b (mon.1) 172 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:04.839744+0000 mon.a (mon.0) 1443 : audit [INF] from='client.48721 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]: dispatch 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:04.840406+0000 mon.a (mon.0) 1444 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:05.840595+0000 mon.a (mon.0) 1445 : audit [INF] from='client.48721 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi105-61649-30"}]': finished 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: audit 2024-02-18T01:25:05.840737+0000 mon.a (mon.0) 1446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:05 smithi155 bash[20805]: cluster 2024-02-18T01:25:05.840871+0000 mon.a (mon.0) 1447 : cluster [DBG] osdmap e180: 8 total, 8 up, 8 in 2024-02-18T01:25:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:06] "GET /metrics HTTP/1.1" 200 188077 "" "Prometheus/2.33.4" 2024-02-18T01:25:06.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: cluster 2024-02-18T01:25:05.668970+0000 mgr.y (mgr.24475) 333 : cluster [DBG] pgmap v346: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:06.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.841812+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.842355+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.105:0/1555630275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.843510+0000 mon.a (mon.0) 1448 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.843915+0000 mon.a (mon.0) 1449 : audit [INF] from='client.47987 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.867290+0000 mon.c (mon.2) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.867884+0000 mon.a (mon.0) 1450 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.868788+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.869056+0000 mon.a (mon.0) 1451 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.869499+0000 mon.c (mon.2) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:05.869775+0000 mon.a (mon.0) 1452 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.505802+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: cluster 2024-02-18T01:25:06.831424+0000 mon.a (mon.0) 1454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.841663+0000 mon.a (mon.0) 1455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]': finished 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.841859+0000 mon.a (mon.0) 1456 : audit [INF] from='client.47987 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: cluster 2024-02-18T01:25:05.668970+0000 mgr.y (mgr.24475) 333 : cluster [DBG] pgmap v346: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.841812+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:06.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.842355+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.105:0/1555630275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.843510+0000 mon.a (mon.0) 1448 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.843915+0000 mon.a (mon.0) 1449 : audit [INF] from='client.47987 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.867290+0000 mon.c (mon.2) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.867884+0000 mon.a (mon.0) 1450 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.868788+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.869056+0000 mon.a (mon.0) 1451 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.869499+0000 mon.c (mon.2) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:05.869775+0000 mon.a (mon.0) 1452 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.505802+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: cluster 2024-02-18T01:25:06.831424+0000 mon.a (mon.0) 1454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.841663+0000 mon.a (mon.0) 1455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]': finished 2024-02-18T01:25:06.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.841859+0000 mon.a (mon.0) 1456 : audit [INF] from='client.47987 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.842195+0000 mon.a (mon.0) 1457 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: cluster 2024-02-18T01:25:06.842400+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.844977+0000 mon.c (mon.2) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[27669]: audit 2024-02-18T01:25:06.849931+0000 mon.a (mon.0) 1459 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.842195+0000 mon.a (mon.0) 1457 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: cluster 2024-02-18T01:25:06.842400+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.844977+0000 mon.c (mon.2) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:06.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:06 smithi105 bash[19734]: audit 2024-02-18T01:25:06.849931+0000 mon.a (mon.0) 1459 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: cluster 2024-02-18T01:25:05.668970+0000 mgr.y (mgr.24475) 333 : cluster [DBG] pgmap v346: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.841812+0000 mon.b (mon.1) 173 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.842355+0000 mon.c (mon.2) 280 : audit [INF] from='client.? v1:172.21.15.105:0/1555630275' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.843510+0000 mon.a (mon.0) 1448 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.843915+0000 mon.a (mon.0) 1449 : audit [INF] from='client.47987 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.867290+0000 mon.c (mon.2) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.867884+0000 mon.a (mon.0) 1450 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.868788+0000 mon.c (mon.2) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.869056+0000 mon.a (mon.0) 1451 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.869499+0000 mon.c (mon.2) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:05.869775+0000 mon.a (mon.0) 1452 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:07.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.505802+0000 mon.a (mon.0) 1453 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: cluster 2024-02-18T01:25:06.831424+0000 mon.a (mon.0) 1454 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.841663+0000 mon.a (mon.0) 1455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-19", "mode": "writeback"}]': finished 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.841859+0000 mon.a (mon.0) 1456 : audit [INF] from='client.47987 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi105-61657-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.842195+0000 mon.a (mon.0) 1457 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi105-61649-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: cluster 2024-02-18T01:25:06.842400+0000 mon.a (mon.0) 1458 : cluster [DBG] osdmap e181: 8 total, 8 up, 8 in 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.844977+0000 mon.c (mon.2) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:06 smithi155 bash[20805]: audit 2024-02-18T01:25:06.849931+0000 mon.a (mon.0) 1459 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:07.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:08 smithi105 bash[19734]: audit 2024-02-18T01:25:06.890528+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:08.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:08 smithi105 bash[19734]: audit 2024-02-18T01:25:06.890880+0000 mon.a (mon.0) 1460 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:08 smithi105 bash[27669]: audit 2024-02-18T01:25:06.890528+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:08.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:08 smithi105 bash[27669]: audit 2024-02-18T01:25:06.890880+0000 mon.a (mon.0) 1460 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:08 smithi155 bash[20805]: audit 2024-02-18T01:25:06.890528+0000 mon.b (mon.1) 174 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:08.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:08 smithi155 bash[20805]: audit 2024-02-18T01:25:06.890880+0000 mon.a (mon.0) 1460 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:09.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:07.665428+0000 mgr.y (mgr.24475) 334 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: cluster 2024-02-18T01:25:07.671067+0000 mgr.y (mgr.24475) 335 : cluster [DBG] pgmap v349: 260 pgs: 260 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.264503+0000 mon.a (mon.0) 1461 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: cluster 2024-02-18T01:25:08.264707+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.265988+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.267057+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: cluster 2024-02-18T01:25:08.838964+0000 mon.a (mon.0) 1464 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.850718+0000 mon.a (mon.0) 1465 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.850994+0000 mon.a (mon.0) 1466 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: cluster 2024-02-18T01:25:08.851262+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.856212+0000 mon.c (mon.2) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2397605045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:09.744 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[19734]: audit 2024-02-18T01:25:08.858827+0000 mon.a (mon.0) 1468 : audit [INF] from='client.47999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:07.665428+0000 mgr.y (mgr.24475) 334 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: cluster 2024-02-18T01:25:07.671067+0000 mgr.y (mgr.24475) 335 : cluster [DBG] pgmap v349: 260 pgs: 260 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.264503+0000 mon.a (mon.0) 1461 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: cluster 2024-02-18T01:25:08.264707+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.265988+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.267057+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: cluster 2024-02-18T01:25:08.838964+0000 mon.a (mon.0) 1464 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.850718+0000 mon.a (mon.0) 1465 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.850994+0000 mon.a (mon.0) 1466 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:09.745 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: cluster 2024-02-18T01:25:08.851262+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.856212+0000 mon.c (mon.2) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2397605045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:09 smithi105 bash[27669]: audit 2024-02-18T01:25:08.858827+0000 mon.a (mon.0) 1468 : audit [INF] from='client.47999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:07.665428+0000 mgr.y (mgr.24475) 334 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: cluster 2024-02-18T01:25:07.671067+0000 mgr.y (mgr.24475) 335 : cluster [DBG] pgmap v349: 260 pgs: 260 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.264503+0000 mon.a (mon.0) 1461 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: cluster 2024-02-18T01:25:08.264707+0000 mon.a (mon.0) 1462 : cluster [DBG] osdmap e182: 8 total, 8 up, 8 in 2024-02-18T01:25:09.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.265988+0000 mon.b (mon.1) 175 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.267057+0000 mon.a (mon.0) 1463 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]: dispatch 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: cluster 2024-02-18T01:25:08.838964+0000 mon.a (mon.0) 1464 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.850718+0000 mon.a (mon.0) 1465 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi105-61649-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.850994+0000 mon.a (mon.0) 1466 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-19"}]': finished 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: cluster 2024-02-18T01:25:08.851262+0000 mon.a (mon.0) 1467 : cluster [DBG] osdmap e183: 8 total, 8 up, 8 in 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.856212+0000 mon.c (mon.2) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2397605045' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:09.747 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:09 smithi155 bash[20805]: audit 2024-02-18T01:25:08.858827+0000 mon.a (mon.0) 1468 : audit [INF] from='client.47999 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[27669]: cluster 2024-02-18T01:25:09.671917+0000 mgr.y (mgr.24475) 336 : cluster [DBG] pgmap v352: 268 pgs: 40 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:25:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[27669]: audit 2024-02-18T01:25:09.852897+0000 mon.a (mon.0) 1469 : audit [INF] from='client.47999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[27669]: cluster 2024-02-18T01:25:09.853472+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-18T01:25:11.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[19734]: cluster 2024-02-18T01:25:09.671917+0000 mgr.y (mgr.24475) 336 : cluster [DBG] pgmap v352: 268 pgs: 40 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:25:11.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[19734]: audit 2024-02-18T01:25:09.852897+0000 mon.a (mon.0) 1469 : audit [INF] from='client.47999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:11.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:10 smithi105 bash[19734]: cluster 2024-02-18T01:25:09.853472+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-18T01:25:11.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:10 smithi155 bash[20805]: cluster 2024-02-18T01:25:09.671917+0000 mgr.y (mgr.24475) 336 : cluster [DBG] pgmap v352: 268 pgs: 40 unknown, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-02-18T01:25:11.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:10 smithi155 bash[20805]: audit 2024-02-18T01:25:09.852897+0000 mon.a (mon.0) 1469 : audit [INF] from='client.47999 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi105-61657-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:11.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:10 smithi155 bash[20805]: cluster 2024-02-18T01:25:09.853472+0000 mon.a (mon.0) 1470 : cluster [DBG] osdmap e184: 8 total, 8 up, 8 in 2024-02-18T01:25:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: cluster 2024-02-18T01:25:10.860092+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-18T01:25:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:10.863207+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:10.863226+0000 mon.c (mon.2) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:10.863772+0000 mon.a (mon.0) 1472 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:10.866943+0000 mon.a (mon.0) 1473 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.650733+0000 mon.a (mon.0) 1474 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.856375+0000 mon.a (mon.0) 1475 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.856457+0000 mon.a (mon.0) 1476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: cluster 2024-02-18T01:25:11.856628+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.857135+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.857704+0000 mon.a (mon.0) 1478 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.858712+0000 mon.c (mon.2) 288 : audit [INF] from='client.? v1:172.21.15.105:0/3435399565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[19734]: audit 2024-02-18T01:25:11.860610+0000 mon.a (mon.0) 1479 : audit [INF] from='client.48005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: cluster 2024-02-18T01:25:10.860092+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:10.863207+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:10.863226+0000 mon.c (mon.2) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:10.863772+0000 mon.a (mon.0) 1472 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:10.866943+0000 mon.a (mon.0) 1473 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.650733+0000 mon.a (mon.0) 1474 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.856375+0000 mon.a (mon.0) 1475 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.856457+0000 mon.a (mon.0) 1476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: cluster 2024-02-18T01:25:11.856628+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.857135+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.857704+0000 mon.a (mon.0) 1478 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.858712+0000 mon.c (mon.2) 288 : audit [INF] from='client.? v1:172.21.15.105:0/3435399565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:11 smithi105 bash[27669]: audit 2024-02-18T01:25:11.860610+0000 mon.a (mon.0) 1479 : audit [INF] from='client.48005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: cluster 2024-02-18T01:25:10.860092+0000 mon.a (mon.0) 1471 : cluster [DBG] osdmap e185: 8 total, 8 up, 8 in 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:10.863207+0000 mon.b (mon.1) 176 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:10.863226+0000 mon.c (mon.2) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:10.863772+0000 mon.a (mon.0) 1472 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:10.866943+0000 mon.a (mon.0) 1473 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.650733+0000 mon.a (mon.0) 1474 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.856375+0000 mon.a (mon.0) 1475 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.856457+0000 mon.a (mon.0) 1476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: cluster 2024-02-18T01:25:11.856628+0000 mon.a (mon.0) 1477 : cluster [DBG] osdmap e186: 8 total, 8 up, 8 in 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.857135+0000 mon.c (mon.2) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2473855782' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.857704+0000 mon.a (mon.0) 1478 : audit [INF] from='client.47993 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]: dispatch 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.858712+0000 mon.c (mon.2) 288 : audit [INF] from='client.? v1:172.21.15.105:0/3435399565' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:12.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:11 smithi155 bash[20805]: audit 2024-02-18T01:25:11.860610+0000 mon.a (mon.0) 1479 : audit [INF] from='client.48005 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: cluster 2024-02-18T01:25:11.672564+0000 mgr.y (mgr.24475) 337 : cluster [DBG] pgmap v355: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:11.888780+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:11.889587+0000 mon.a (mon.0) 1480 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:11.966106+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:25:13.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:11.967669+0000 mon.a (mon.0) 1482 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:11.977107+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:12.862925+0000 mon.a (mon.0) 1484 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:12.863042+0000 mon.a (mon.0) 1485 : audit [INF] from='client.48005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:12.863466+0000 mon.a (mon.0) 1486 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: cluster 2024-02-18T01:25:12.863564+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-18T01:25:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:12.864538+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:13.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[19734]: audit 2024-02-18T01:25:12.865060+0000 mon.a (mon.0) 1488 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: cluster 2024-02-18T01:25:11.672564+0000 mgr.y (mgr.24475) 337 : cluster [DBG] pgmap v355: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:13.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:11.888780+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:11.889587+0000 mon.a (mon.0) 1480 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:11.966106+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:25:13.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:11.967669+0000 mon.a (mon.0) 1482 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:25:13.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:11.977107+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:13.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:12.862925+0000 mon.a (mon.0) 1484 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:13.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:12.863042+0000 mon.a (mon.0) 1485 : audit [INF] from='client.48005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:13.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:12.863466+0000 mon.a (mon.0) 1486 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:13.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: cluster 2024-02-18T01:25:12.863564+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-18T01:25:13.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:12.864538+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:13.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:12 smithi105 bash[27669]: audit 2024-02-18T01:25:12.865060+0000 mon.a (mon.0) 1488 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: cluster 2024-02-18T01:25:11.672564+0000 mgr.y (mgr.24475) 337 : cluster [DBG] pgmap v355: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:11.888780+0000 mon.b (mon.1) 177 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:11.889587+0000 mon.a (mon.0) 1480 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:11.966106+0000 mon.a (mon.0) 1481 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:25:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:11.967669+0000 mon.a (mon.0) 1482 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:25:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:11.977107+0000 mon.a (mon.0) 1483 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:25:13.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:12.862925+0000 mon.a (mon.0) 1484 : audit [INF] from='client.47993 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi105-61649-31"}]': finished 2024-02-18T01:25:13.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:12.863042+0000 mon.a (mon.0) 1485 : audit [INF] from='client.48005 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:13.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:12.863466+0000 mon.a (mon.0) 1486 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:13.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: cluster 2024-02-18T01:25:12.863564+0000 mon.a (mon.0) 1487 : cluster [DBG] osdmap e187: 8 total, 8 up, 8 in 2024-02-18T01:25:13.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:12.864538+0000 mon.b (mon.1) 178 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:13.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:12 smithi155 bash[20805]: audit 2024-02-18T01:25:12.865060+0000 mon.a (mon.0) 1488 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.900036+0000 mon.c (mon.2) 289 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.900719+0000 mon.a (mon.0) 1489 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.901475+0000 mon.c (mon.2) 290 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.901788+0000 mon.a (mon.0) 1490 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.902269+0000 mon.c (mon.2) 291 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:12.902579+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.867104+0000 mon.a (mon.0) 1492 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.867752+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: cluster 2024-02-18T01:25:13.868335+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-18T01:25:14.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.869155+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.870558+0000 mon.a (mon.0) 1495 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.871574+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.872015+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.105:0/4147996157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.872614+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[19734]: audit 2024-02-18T01:25:13.873478+0000 mon.a (mon.0) 1497 : audit [INF] from='client.48017 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.900036+0000 mon.c (mon.2) 289 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.900719+0000 mon.a (mon.0) 1489 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.901475+0000 mon.c (mon.2) 290 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.901788+0000 mon.a (mon.0) 1490 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.902269+0000 mon.c (mon.2) 291 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:12.902579+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.867104+0000 mon.a (mon.0) 1492 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.867752+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: cluster 2024-02-18T01:25:13.868335+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.869155+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.870558+0000 mon.a (mon.0) 1495 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.871574+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.872015+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.105:0/4147996157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.872614+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:13 smithi105 bash[27669]: audit 2024-02-18T01:25:13.873478+0000 mon.a (mon.0) 1497 : audit [INF] from='client.48017 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.900036+0000 mon.c (mon.2) 289 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.900719+0000 mon.a (mon.0) 1489 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.901475+0000 mon.c (mon.2) 290 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.901788+0000 mon.a (mon.0) 1490 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.902269+0000 mon.c (mon.2) 291 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:12.902579+0000 mon.a (mon.0) 1491 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.867104+0000 mon.a (mon.0) 1492 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.867752+0000 mon.a (mon.0) 1493 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi105-61649-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: cluster 2024-02-18T01:25:13.868335+0000 mon.a (mon.0) 1494 : cluster [DBG] osdmap e188: 8 total, 8 up, 8 in 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.869155+0000 mon.b (mon.1) 179 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.870558+0000 mon.a (mon.0) 1495 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.871574+0000 mon.c (mon.2) 292 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.872015+0000 mon.c (mon.2) 293 : audit [INF] from='client.? v1:172.21.15.105:0/4147996157' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.872614+0000 mon.a (mon.0) 1496 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:14.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:13 smithi155 bash[20805]: audit 2024-02-18T01:25:13.873478+0000 mon.a (mon.0) 1497 : audit [INF] from='client.48017 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:15.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:15 smithi105 bash[19734]: cluster 2024-02-18T01:25:13.673429+0000 mgr.y (mgr.24475) 338 : cluster [DBG] pgmap v358: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:25:15.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:15 smithi105 bash[19734]: cluster 2024-02-18T01:25:14.860137+0000 mon.a (mon.0) 1498 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:15.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:15 smithi105 bash[27669]: cluster 2024-02-18T01:25:13.673429+0000 mgr.y (mgr.24475) 338 : cluster [DBG] pgmap v358: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:25:15.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:15 smithi105 bash[27669]: cluster 2024-02-18T01:25:14.860137+0000 mon.a (mon.0) 1498 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:15.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:15 smithi155 bash[20805]: cluster 2024-02-18T01:25:13.673429+0000 mgr.y (mgr.24475) 338 : cluster [DBG] pgmap v358: 260 pgs: 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:25:15.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:15 smithi155 bash[20805]: cluster 2024-02-18T01:25:14.860137+0000 mon.a (mon.0) 1498 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:14.868360+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]': finished 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:14.940062+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.284917+0000 mon.a (mon.0) 1500 : audit [INF] from='client.48017 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: cluster 2024-02-18T01:25:15.285346+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.292957+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: cluster 2024-02-18T01:25:15.674242+0000 mgr.y (mgr.24475) 339 : cluster [DBG] pgmap v361: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.875173+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.875277+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: cluster 2024-02-18T01:25:15.875367+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.877427+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.878731+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.880374+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.105:0/43410658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[19734]: audit 2024-02-18T01:25:15.881774+0000 mon.a (mon.0) 1507 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:14.868360+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]': finished 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:14.940062+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.284917+0000 mon.a (mon.0) 1500 : audit [INF] from='client.48017 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: cluster 2024-02-18T01:25:15.285346+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.292957+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: cluster 2024-02-18T01:25:15.674242+0000 mgr.y (mgr.24475) 339 : cluster [DBG] pgmap v361: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.875173+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.875277+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: cluster 2024-02-18T01:25:15.875367+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.877427+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.878731+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.880374+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.105:0/43410658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[27669]: audit 2024-02-18T01:25:15.881774+0000 mon.a (mon.0) 1507 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:14.868360+0000 mon.a (mon.0) 1499 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-21", "mode": "writeback"}]': finished 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:14.940062+0000 mon.b (mon.1) 180 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.284917+0000 mon.a (mon.0) 1500 : audit [INF] from='client.48017 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: cluster 2024-02-18T01:25:15.285346+0000 mon.a (mon.0) 1501 : cluster [DBG] osdmap e189: 8 total, 8 up, 8 in 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.292957+0000 mon.a (mon.0) 1502 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: cluster 2024-02-18T01:25:15.674242+0000 mgr.y (mgr.24475) 339 : cluster [DBG] pgmap v361: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.875173+0000 mon.a (mon.0) 1503 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi105-61649-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.875277+0000 mon.a (mon.0) 1504 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: cluster 2024-02-18T01:25:15.875367+0000 mon.a (mon.0) 1505 : cluster [DBG] osdmap e190: 8 total, 8 up, 8 in 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.877427+0000 mon.b (mon.1) 181 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.878731+0000 mon.a (mon.0) 1506 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]: dispatch 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.880374+0000 mon.c (mon.2) 294 : audit [INF] from='client.? v1:172.21.15.105:0/43410658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:16 smithi155 bash[20805]: audit 2024-02-18T01:25:15.881774+0000 mon.a (mon.0) 1507 : audit [INF] from='client.48023 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:16] "GET /metrics HTTP/1.1" 200 188077 "" "Prometheus/2.33.4" 2024-02-18T01:25:17.679 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:17 smithi155 bash[20805]: cluster 2024-02-18T01:25:16.868092+0000 mon.a (mon.0) 1508 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:17.679 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:17 smithi155 bash[20805]: audit 2024-02-18T01:25:16.874448+0000 mon.a (mon.0) 1509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:17.679 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:17 smithi155 bash[20805]: audit 2024-02-18T01:25:16.874695+0000 mon.a (mon.0) 1510 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:17.679 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:17 smithi155 bash[20805]: cluster 2024-02-18T01:25:16.874855+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-18T01:25:17.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[19734]: cluster 2024-02-18T01:25:16.868092+0000 mon.a (mon.0) 1508 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:17.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[19734]: audit 2024-02-18T01:25:16.874448+0000 mon.a (mon.0) 1509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:17.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[19734]: audit 2024-02-18T01:25:16.874695+0000 mon.a (mon.0) 1510 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:17.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[19734]: cluster 2024-02-18T01:25:16.874855+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-18T01:25:17.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[27669]: cluster 2024-02-18T01:25:16.868092+0000 mon.a (mon.0) 1508 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:17.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[27669]: audit 2024-02-18T01:25:16.874448+0000 mon.a (mon.0) 1509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-21"}]': finished 2024-02-18T01:25:17.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[27669]: audit 2024-02-18T01:25:16.874695+0000 mon.a (mon.0) 1510 : audit [INF] from='client.48023 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi105-61657-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:17.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:17 smithi105 bash[27669]: cluster 2024-02-18T01:25:16.874855+0000 mon.a (mon.0) 1511 : cluster [DBG] osdmap e191: 8 total, 8 up, 8 in 2024-02-18T01:25:17.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: cluster 2024-02-18T01:25:17.677008+0000 mgr.y (mgr.24475) 340 : cluster [DBG] pgmap v364: 332 pgs: 332 active+clean; 7.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-18T01:25:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: audit 2024-02-18T01:25:17.677290+0000 mgr.y (mgr.24475) 341 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: cluster 2024-02-18T01:25:17.881005+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-18T01:25:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: audit 2024-02-18T01:25:17.884683+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: cluster 2024-02-18T01:25:18.324546+0000 mon.a (mon.0) 1513 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[19734]: audit 2024-02-18T01:25:18.325335+0000 mon.a (mon.0) 1514 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: cluster 2024-02-18T01:25:17.677008+0000 mgr.y (mgr.24475) 340 : cluster [DBG] pgmap v364: 332 pgs: 332 active+clean; 7.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: audit 2024-02-18T01:25:17.677290+0000 mgr.y (mgr.24475) 341 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: cluster 2024-02-18T01:25:17.881005+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: audit 2024-02-18T01:25:17.884683+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: cluster 2024-02-18T01:25:18.324546+0000 mon.a (mon.0) 1513 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:19.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:18 smithi105 bash[27669]: audit 2024-02-18T01:25:18.325335+0000 mon.a (mon.0) 1514 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: cluster 2024-02-18T01:25:17.677008+0000 mgr.y (mgr.24475) 340 : cluster [DBG] pgmap v364: 332 pgs: 332 active+clean; 7.0 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 767 B/s wr, 4 op/s 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: audit 2024-02-18T01:25:17.677290+0000 mgr.y (mgr.24475) 341 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: cluster 2024-02-18T01:25:17.881005+0000 mon.a (mon.0) 1512 : cluster [DBG] osdmap e192: 8 total, 8 up, 8 in 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: audit 2024-02-18T01:25:17.884683+0000 mon.c (mon.2) 295 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: cluster 2024-02-18T01:25:18.324546+0000 mon.a (mon.0) 1513 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:18 smithi155 bash[20805]: audit 2024-02-18T01:25:18.325335+0000 mon.a (mon.0) 1514 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:18.885125+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: cluster 2024-02-18T01:25:18.885264+0000 mon.a (mon.0) 1516 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-18T01:25:20.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:18.890512+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:18.890524+0000 mon.c (mon.2) 296 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:18.892970+0000 mon.a (mon.0) 1517 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:18.893580+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:19.887193+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: audit 2024-02-18T01:25:19.887542+0000 mon.a (mon.0) 1520 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[19734]: cluster 2024-02-18T01:25:19.887764+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:18.885125+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: cluster 2024-02-18T01:25:18.885264+0000 mon.a (mon.0) 1516 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:18.890512+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:18.890524+0000 mon.c (mon.2) 296 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:18.892970+0000 mon.a (mon.0) 1517 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:18.893580+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:19.887193+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: audit 2024-02-18T01:25:19.887542+0000 mon.a (mon.0) 1520 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:20.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:19 smithi105 bash[27669]: cluster 2024-02-18T01:25:19.887764+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:18.885125+0000 mon.a (mon.0) 1515 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: cluster 2024-02-18T01:25:18.885264+0000 mon.a (mon.0) 1516 : cluster [DBG] osdmap e193: 8 total, 8 up, 8 in 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:18.890512+0000 mon.b (mon.1) 182 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:18.890524+0000 mon.c (mon.2) 296 : audit [INF] from='client.? v1:172.21.15.105:0/3674796807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:18.892970+0000 mon.a (mon.0) 1517 : audit [INF] from='client.48011 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]: dispatch 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:18.893580+0000 mon.a (mon.0) 1518 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:19.887193+0000 mon.a (mon.0) 1519 : audit [INF] from='client.48011 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi105-61649-32"}]': finished 2024-02-18T01:25:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: audit 2024-02-18T01:25:19.887542+0000 mon.a (mon.0) 1520 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:19 smithi155 bash[20805]: cluster 2024-02-18T01:25:19.887764+0000 mon.a (mon.0) 1521 : cluster [DBG] osdmap e194: 8 total, 8 up, 8 in 2024-02-18T01:25:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: cluster 2024-02-18T01:25:19.677828+0000 mgr.y (mgr.24475) 342 : cluster [DBG] pgmap v367: 260 pgs: 32 unknown, 228 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:19.936764+0000 mon.c (mon.2) 297 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:20.377177+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:20.378176+0000 mon.c (mon.2) 298 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:20.378555+0000 mon.a (mon.0) 1523 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:20.379249+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[19734]: audit 2024-02-18T01:25:20.379658+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: cluster 2024-02-18T01:25:19.677828+0000 mgr.y (mgr.24475) 342 : cluster [DBG] pgmap v367: 260 pgs: 32 unknown, 228 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:19.936764+0000 mon.c (mon.2) 297 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:20.377177+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:20.378176+0000 mon.c (mon.2) 298 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:20.378555+0000 mon.a (mon.0) 1523 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:20.379249+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:21.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:20 smithi105 bash[27669]: audit 2024-02-18T01:25:20.379658+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: cluster 2024-02-18T01:25:19.677828+0000 mgr.y (mgr.24475) 342 : cluster [DBG] pgmap v367: 260 pgs: 32 unknown, 228 active+clean; 6.9 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:19.936764+0000 mon.c (mon.2) 297 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:20.377177+0000 mon.a (mon.0) 1522 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:20.378176+0000 mon.c (mon.2) 298 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:20.378555+0000 mon.a (mon.0) 1523 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:20.379249+0000 mon.c (mon.2) 299 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:21.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:20 smithi155 bash[20805]: audit 2024-02-18T01:25:20.379658+0000 mon.a (mon.0) 1524 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:22.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.911310+0000 mon.a (mon.0) 1525 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:22.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: cluster 2024-02-18T01:25:20.911453+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-18T01:25:22.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.912918+0000 mon.c (mon.2) 300 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.913989+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.916892+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.930439+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[27669]: audit 2024-02-18T01:25:20.930731+0000 mon.a (mon.0) 1529 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.911310+0000 mon.a (mon.0) 1525 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: cluster 2024-02-18T01:25:20.911453+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.912918+0000 mon.c (mon.2) 300 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.913989+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.916892+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.930439+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:22.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:22 smithi105 bash[19734]: audit 2024-02-18T01:25:20.930731+0000 mon.a (mon.0) 1529 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.911310+0000 mon.a (mon.0) 1525 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi105-61649-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: cluster 2024-02-18T01:25:20.911453+0000 mon.a (mon.0) 1526 : cluster [DBG] osdmap e195: 8 total, 8 up, 8 in 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.912918+0000 mon.c (mon.2) 300 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.913989+0000 mon.a (mon.0) 1527 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.916892+0000 mon.a (mon.0) 1528 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.930439+0000 mon.b (mon.1) 183 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:22 smithi155 bash[20805]: audit 2024-02-18T01:25:20.930731+0000 mon.a (mon.0) 1529 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:23.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: cluster 2024-02-18T01:25:21.678694+0000 mgr.y (mgr.24475) 343 : cluster [DBG] pgmap v370: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:23.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.340608+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.341074+0000 mon.a (mon.0) 1531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: cluster 2024-02-18T01:25:22.341290+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.342326+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.352580+0000 mon.a (mon.0) 1533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.918798+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.918933+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: cluster 2024-02-18T01:25:22.919079+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.920747+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.922993+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.105:0/1382816133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.925480+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[19734]: audit 2024-02-18T01:25:22.926377+0000 mon.a (mon.0) 1538 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: cluster 2024-02-18T01:25:21.678694+0000 mgr.y (mgr.24475) 343 : cluster [DBG] pgmap v370: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.340608+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.341074+0000 mon.a (mon.0) 1531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: cluster 2024-02-18T01:25:22.341290+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.342326+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.352580+0000 mon.a (mon.0) 1533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.918798+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.918933+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: cluster 2024-02-18T01:25:22.919079+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.920747+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.922993+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.105:0/1382816133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.925480+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:23 smithi105 bash[27669]: audit 2024-02-18T01:25:22.926377+0000 mon.a (mon.0) 1538 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: cluster 2024-02-18T01:25:21.678694+0000 mgr.y (mgr.24475) 343 : cluster [DBG] pgmap v370: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.340608+0000 mon.a (mon.0) 1530 : audit [INF] from='client.? v1:172.21.15.105:0/3199126167' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.341074+0000 mon.a (mon.0) 1531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: cluster 2024-02-18T01:25:22.341290+0000 mon.a (mon.0) 1532 : cluster [DBG] osdmap e196: 8 total, 8 up, 8 in 2024-02-18T01:25:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.342326+0000 mon.b (mon.1) 184 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.352580+0000 mon.a (mon.0) 1533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.918798+0000 mon.a (mon.0) 1534 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi105-61649-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.918933+0000 mon.a (mon.0) 1535 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: cluster 2024-02-18T01:25:22.919079+0000 mon.a (mon.0) 1536 : cluster [DBG] osdmap e197: 8 total, 8 up, 8 in 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.920747+0000 mon.b (mon.1) 185 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.922993+0000 mon.c (mon.2) 301 : audit [INF] from='client.? v1:172.21.15.105:0/1382816133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.925480+0000 mon.a (mon.0) 1537 : audit [INF] from='client.48041 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:23.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:23 smithi155 bash[20805]: audit 2024-02-18T01:25:22.926377+0000 mon.a (mon.0) 1538 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]: dispatch 2024-02-18T01:25:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[19734]: cluster 2024-02-18T01:25:23.812316+0000 mon.a (mon.0) 1539 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[19734]: cluster 2024-02-18T01:25:23.910881+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[19734]: audit 2024-02-18T01:25:23.921096+0000 mon.a (mon.0) 1541 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[19734]: audit 2024-02-18T01:25:23.921277+0000 mon.a (mon.0) 1542 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]': finished 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[19734]: cluster 2024-02-18T01:25:23.921744+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[27669]: cluster 2024-02-18T01:25:23.812316+0000 mon.a (mon.0) 1539 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[27669]: cluster 2024-02-18T01:25:23.910881+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[27669]: audit 2024-02-18T01:25:23.921096+0000 mon.a (mon.0) 1541 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[27669]: audit 2024-02-18T01:25:23.921277+0000 mon.a (mon.0) 1542 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]': finished 2024-02-18T01:25:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:24 smithi105 bash[27669]: cluster 2024-02-18T01:25:23.921744+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-18T01:25:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:24 smithi155 bash[20805]: cluster 2024-02-18T01:25:23.812316+0000 mon.a (mon.0) 1539 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:25:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:24 smithi155 bash[20805]: cluster 2024-02-18T01:25:23.910881+0000 mon.a (mon.0) 1540 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:24 smithi155 bash[20805]: audit 2024-02-18T01:25:23.921096+0000 mon.a (mon.0) 1541 : audit [INF] from='client.48041 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:24 smithi155 bash[20805]: audit 2024-02-18T01:25:23.921277+0000 mon.a (mon.0) 1542 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-23", "mode": "writeback"}]': finished 2024-02-18T01:25:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:24 smithi155 bash[20805]: cluster 2024-02-18T01:25:23.921744+0000 mon.a (mon.0) 1543 : cluster [DBG] osdmap e198: 8 total, 8 up, 8 in 2024-02-18T01:25:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: cluster 2024-02-18T01:25:23.679602+0000 mgr.y (mgr.24475) 344 : cluster [DBG] pgmap v373: 300 pgs: 40 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:25:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: cluster 2024-02-18T01:25:24.925912+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-18T01:25:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.932185+0000 mon.c (mon.2) 302 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.932593+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.105:0/1999573491' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.933058+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.933587+0000 mon.a (mon.0) 1546 : audit [INF] from='client.48047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.961656+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[19734]: audit 2024-02-18T01:25:24.961996+0000 mon.a (mon.0) 1547 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: cluster 2024-02-18T01:25:23.679602+0000 mgr.y (mgr.24475) 344 : cluster [DBG] pgmap v373: 300 pgs: 40 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: cluster 2024-02-18T01:25:24.925912+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.932185+0000 mon.c (mon.2) 302 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.932593+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.105:0/1999573491' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.933058+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.933587+0000 mon.a (mon.0) 1546 : audit [INF] from='client.48047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.961656+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:25.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:25 smithi105 bash[27669]: audit 2024-02-18T01:25:24.961996+0000 mon.a (mon.0) 1547 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: cluster 2024-02-18T01:25:23.679602+0000 mgr.y (mgr.24475) 344 : cluster [DBG] pgmap v373: 300 pgs: 40 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: cluster 2024-02-18T01:25:24.925912+0000 mon.a (mon.0) 1544 : cluster [DBG] osdmap e199: 8 total, 8 up, 8 in 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.932185+0000 mon.c (mon.2) 302 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.932593+0000 mon.c (mon.2) 303 : audit [INF] from='client.? v1:172.21.15.105:0/1999573491' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.933058+0000 mon.a (mon.0) 1545 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.933587+0000 mon.a (mon.0) 1546 : audit [INF] from='client.48047 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:25.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.961656+0000 mon.b (mon.1) 186 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:25.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:25 smithi155 bash[20805]: audit 2024-02-18T01:25:24.961996+0000 mon.a (mon.0) 1547 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:26.948 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:26] "GET /metrics HTTP/1.1" 200 188322 "" "Prometheus/2.33.4" 2024-02-18T01:25:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: cluster 2024-02-18T01:25:25.680516+0000 mgr.y (mgr.24475) 345 : cluster [DBG] pgmap v376: 324 pgs: 64 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.930269+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.930532+0000 mon.a (mon.0) 1549 : audit [INF] from='client.48047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.931605+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.931647+0000 mon.a (mon.0) 1550 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: cluster 2024-02-18T01:25:25.931939+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.933355+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.933514+0000 mon.a (mon.0) 1552 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[19734]: audit 2024-02-18T01:25:25.935278+0000 mon.a (mon.0) 1553 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: cluster 2024-02-18T01:25:25.680516+0000 mgr.y (mgr.24475) 345 : cluster [DBG] pgmap v376: 324 pgs: 64 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.930269+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:27.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.930532+0000 mon.a (mon.0) 1549 : audit [INF] from='client.48047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.931605+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.931647+0000 mon.a (mon.0) 1550 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: cluster 2024-02-18T01:25:25.931939+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.933355+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.933514+0000 mon.a (mon.0) 1552 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:26 smithi105 bash[27669]: audit 2024-02-18T01:25:25.935278+0000 mon.a (mon.0) 1553 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: cluster 2024-02-18T01:25:25.680516+0000 mgr.y (mgr.24475) 345 : cluster [DBG] pgmap v376: 324 pgs: 64 unknown, 14 creating+peering, 11 creating+activating, 235 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.930269+0000 mon.a (mon.0) 1548 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.930532+0000 mon.a (mon.0) 1549 : audit [INF] from='client.48047 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.931605+0000 mon.c (mon.2) 304 : audit [INF] from='client.? v1:172.21.15.105:0/390858467' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.931647+0000 mon.a (mon.0) 1550 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: cluster 2024-02-18T01:25:25.931939+0000 mon.a (mon.0) 1551 : cluster [DBG] osdmap e200: 8 total, 8 up, 8 in 2024-02-18T01:25:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.933355+0000 mon.b (mon.1) 187 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.933514+0000 mon.a (mon.0) 1552 : audit [INF] from='client.48035 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]: dispatch 2024-02-18T01:25:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:26 smithi155 bash[20805]: audit 2024-02-18T01:25:25.935278+0000 mon.a (mon.0) 1553 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]: dispatch 2024-02-18T01:25:27.995 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: cluster 2024-02-18T01:25:26.924127+0000 mon.a (mon.0) 1554 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.934248+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.934634+0000 mon.a (mon.0) 1556 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: cluster 2024-02-18T01:25:26.935265+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.952472+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.961608+0000 mon.c (mon.2) 305 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.961954+0000 mon.a (mon.0) 1559 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.963318+0000 mon.c (mon.2) 306 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.963614+0000 mon.a (mon.0) 1560 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.964136+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:26.964417+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:27.936544+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:27.939069+0000 mon.a (mon.0) 1563 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: cluster 2024-02-18T01:25:27.939674+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[27669]: audit 2024-02-18T01:25:27.940161+0000 mon.c (mon.2) 308 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: cluster 2024-02-18T01:25:26.924127+0000 mon.a (mon.0) 1554 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.934248+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.934634+0000 mon.a (mon.0) 1556 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: cluster 2024-02-18T01:25:26.935265+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.952472+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.961608+0000 mon.c (mon.2) 305 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.961954+0000 mon.a (mon.0) 1559 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.963318+0000 mon.c (mon.2) 306 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.963614+0000 mon.a (mon.0) 1560 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.964136+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:26.964417+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:27.936544+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:27.939069+0000 mon.a (mon.0) 1563 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: cluster 2024-02-18T01:25:27.939674+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-18T01:25:28.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:28 smithi105 bash[19734]: audit 2024-02-18T01:25:27.940161+0000 mon.c (mon.2) 308 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: cluster 2024-02-18T01:25:26.924127+0000 mon.a (mon.0) 1554 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:28.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.934248+0000 mon.a (mon.0) 1555 : audit [INF] from='client.48035 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi105-61649-33"}]': finished 2024-02-18T01:25:28.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.934634+0000 mon.a (mon.0) 1556 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-23"}]': finished 2024-02-18T01:25:28.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: cluster 2024-02-18T01:25:26.935265+0000 mon.a (mon.0) 1557 : cluster [DBG] osdmap e201: 8 total, 8 up, 8 in 2024-02-18T01:25:28.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.952472+0000 mon.a (mon.0) 1558 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.961608+0000 mon.c (mon.2) 305 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.961954+0000 mon.a (mon.0) 1559 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.963318+0000 mon.c (mon.2) 306 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.963614+0000 mon.a (mon.0) 1560 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.964136+0000 mon.c (mon.2) 307 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:26.964417+0000 mon.a (mon.0) 1561 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:27.936544+0000 mon.a (mon.0) 1562 : audit [INF] from='client.? v1:172.21.15.105:0/2479615010' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:27.939069+0000 mon.a (mon.0) 1563 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi105-61649-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: cluster 2024-02-18T01:25:27.939674+0000 mon.a (mon.0) 1564 : cluster [DBG] osdmap e202: 8 total, 8 up, 8 in 2024-02-18T01:25:28.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:28 smithi155 bash[20805]: audit 2024-02-18T01:25:27.940161+0000 mon.c (mon.2) 308 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: cluster 2024-02-18T01:25:27.682444+0000 mgr.y (mgr.24475) 346 : cluster [DBG] pgmap v379: 356 pgs: 32 creating+peering, 324 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-18T01:25:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:27.688485+0000 mgr.y (mgr.24475) 347 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:28.389011+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:28.865109+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: cluster 2024-02-18T01:25:29.274713+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:29.276120+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.105:0/729375770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:29.277047+0000 mon.a (mon.0) 1567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[19734]: audit 2024-02-18T01:25:29.278083+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: cluster 2024-02-18T01:25:27.682444+0000 mgr.y (mgr.24475) 346 : cluster [DBG] pgmap v379: 356 pgs: 32 creating+peering, 324 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:27.688485+0000 mgr.y (mgr.24475) 347 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:28.389011+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:28.865109+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: cluster 2024-02-18T01:25:29.274713+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-18T01:25:29.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:29.276120+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.105:0/729375770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:29.277047+0000 mon.a (mon.0) 1567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:29 smithi105 bash[27669]: audit 2024-02-18T01:25:29.278083+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: cluster 2024-02-18T01:25:27.682444+0000 mgr.y (mgr.24475) 346 : cluster [DBG] pgmap v379: 356 pgs: 32 creating+peering, 324 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 511 B/s wr, 5 op/s 2024-02-18T01:25:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:27.688485+0000 mgr.y (mgr.24475) 347 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:28.389011+0000 mon.a (mon.0) 1565 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:29.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:28.865109+0000 mon.b (mon.1) 188 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: cluster 2024-02-18T01:25:29.274713+0000 mon.a (mon.0) 1566 : cluster [DBG] osdmap e203: 8 total, 8 up, 8 in 2024-02-18T01:25:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:29.276120+0000 mon.c (mon.2) 309 : audit [INF] from='client.? v1:172.21.15.105:0/729375770' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:29.277047+0000 mon.a (mon.0) 1567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:29.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:29 smithi155 bash[20805]: audit 2024-02-18T01:25:29.278083+0000 mon.a (mon.0) 1568 : audit [INF] from='client.48065 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[19734]: cluster 2024-02-18T01:25:29.683342+0000 mgr.y (mgr.24475) 348 : cluster [DBG] pgmap v382: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[19734]: audit 2024-02-18T01:25:29.865997+0000 mon.a (mon.0) 1569 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[19734]: audit 2024-02-18T01:25:29.866380+0000 mon.a (mon.0) 1570 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[19734]: audit 2024-02-18T01:25:29.866635+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[19734]: cluster 2024-02-18T01:25:29.866767+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[27669]: cluster 2024-02-18T01:25:29.683342+0000 mgr.y (mgr.24475) 348 : cluster [DBG] pgmap v382: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[27669]: audit 2024-02-18T01:25:29.865997+0000 mon.a (mon.0) 1569 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[27669]: audit 2024-02-18T01:25:29.866380+0000 mon.a (mon.0) 1570 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[27669]: audit 2024-02-18T01:25:29.866635+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:30 smithi105 bash[27669]: cluster 2024-02-18T01:25:29.866767+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-18T01:25:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:30 smithi155 bash[20805]: cluster 2024-02-18T01:25:29.683342+0000 mgr.y (mgr.24475) 348 : cluster [DBG] pgmap v382: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:30 smithi155 bash[20805]: audit 2024-02-18T01:25:29.865997+0000 mon.a (mon.0) 1569 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi105-61649-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:30 smithi155 bash[20805]: audit 2024-02-18T01:25:29.866380+0000 mon.a (mon.0) 1570 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:30 smithi155 bash[20805]: audit 2024-02-18T01:25:29.866635+0000 mon.a (mon.0) 1571 : audit [INF] from='client.48065 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi105-61657-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:30 smithi155 bash[20805]: cluster 2024-02-18T01:25:29.866767+0000 mon.a (mon.0) 1572 : cluster [DBG] osdmap e204: 8 total, 8 up, 8 in 2024-02-18T01:25:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[19734]: cluster 2024-02-18T01:25:30.865210+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-18T01:25:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[19734]: audit 2024-02-18T01:25:30.892382+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[19734]: audit 2024-02-18T01:25:30.893088+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:32.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[27669]: cluster 2024-02-18T01:25:30.865210+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-18T01:25:32.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[27669]: audit 2024-02-18T01:25:30.892382+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:32.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:31 smithi105 bash[27669]: audit 2024-02-18T01:25:30.893088+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:31 smithi155 bash[20805]: cluster 2024-02-18T01:25:30.865210+0000 mon.a (mon.0) 1573 : cluster [DBG] osdmap e205: 8 total, 8 up, 8 in 2024-02-18T01:25:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:31 smithi155 bash[20805]: audit 2024-02-18T01:25:30.892382+0000 mon.b (mon.1) 189 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:32.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:31 smithi155 bash[20805]: audit 2024-02-18T01:25:30.893088+0000 mon.a (mon.0) 1574 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: cluster 2024-02-18T01:25:31.684274+0000 mgr.y (mgr.24475) 349 : cluster [DBG] pgmap v385: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:31.870843+0000 mon.a (mon.0) 1575 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: cluster 2024-02-18T01:25:31.871223+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:31.872556+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:31.872585+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:31.873472+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:31.874086+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.874266+0000 mon.a (mon.0) 1579 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.874428+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: cluster 2024-02-18T01:25:32.874615+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.875266+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.876006+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.876631+0000 mon.a (mon.0) 1582 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[19734]: audit 2024-02-18T01:25:32.877048+0000 mon.a (mon.0) 1583 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: cluster 2024-02-18T01:25:31.684274+0000 mgr.y (mgr.24475) 349 : cluster [DBG] pgmap v385: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:31.870843+0000 mon.a (mon.0) 1575 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: cluster 2024-02-18T01:25:31.871223+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:31.872556+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:31.872585+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:31.873472+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:31.874086+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.874266+0000 mon.a (mon.0) 1579 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.874428+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: cluster 2024-02-18T01:25:32.874615+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.875266+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.876006+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.876631+0000 mon.a (mon.0) 1582 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:32 smithi105 bash[27669]: audit 2024-02-18T01:25:32.877048+0000 mon.a (mon.0) 1583 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: cluster 2024-02-18T01:25:31.684274+0000 mgr.y (mgr.24475) 349 : cluster [DBG] pgmap v385: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 7.1 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:31.870843+0000 mon.a (mon.0) 1575 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: cluster 2024-02-18T01:25:31.871223+0000 mon.a (mon.0) 1576 : cluster [DBG] osdmap e206: 8 total, 8 up, 8 in 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:31.872556+0000 mon.c (mon.2) 310 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:31.872585+0000 mon.b (mon.1) 190 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:31.873472+0000 mon.a (mon.0) 1577 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:31.874086+0000 mon.a (mon.0) 1578 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.874266+0000 mon.a (mon.0) 1579 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.874428+0000 mon.a (mon.0) 1580 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: cluster 2024-02-18T01:25:32.874615+0000 mon.a (mon.0) 1581 : cluster [DBG] osdmap e207: 8 total, 8 up, 8 in 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.875266+0000 mon.c (mon.2) 311 : audit [INF] from='client.? v1:172.21.15.105:0/3947796818' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.876006+0000 mon.b (mon.1) 191 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.876631+0000 mon.a (mon.0) 1582 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]: dispatch 2024-02-18T01:25:33.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:32 smithi155 bash[20805]: audit 2024-02-18T01:25:32.877048+0000 mon.a (mon.0) 1583 : audit [INF] from='client.48059 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]: dispatch 2024-02-18T01:25:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[19734]: cluster 2024-02-18T01:25:33.866588+0000 mon.a (mon.0) 1584 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[19734]: audit 2024-02-18T01:25:33.875725+0000 mon.a (mon.0) 1585 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]': finished 2024-02-18T01:25:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[19734]: audit 2024-02-18T01:25:33.876060+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[19734]: cluster 2024-02-18T01:25:33.876213+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-18T01:25:34.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[27669]: cluster 2024-02-18T01:25:33.866588+0000 mon.a (mon.0) 1584 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:34.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[27669]: audit 2024-02-18T01:25:33.875725+0000 mon.a (mon.0) 1585 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]': finished 2024-02-18T01:25:34.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[27669]: audit 2024-02-18T01:25:33.876060+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:34.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:33 smithi105 bash[27669]: cluster 2024-02-18T01:25:33.876213+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-18T01:25:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:33 smithi155 bash[20805]: cluster 2024-02-18T01:25:33.866588+0000 mon.a (mon.0) 1584 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:33 smithi155 bash[20805]: audit 2024-02-18T01:25:33.875725+0000 mon.a (mon.0) 1585 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-25", "mode": "writeback"}]': finished 2024-02-18T01:25:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:33 smithi155 bash[20805]: audit 2024-02-18T01:25:33.876060+0000 mon.a (mon.0) 1586 : audit [INF] from='client.48059 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi105-61649-34"}]': finished 2024-02-18T01:25:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:33 smithi155 bash[20805]: cluster 2024-02-18T01:25:33.876213+0000 mon.a (mon.0) 1587 : cluster [DBG] osdmap e208: 8 total, 8 up, 8 in 2024-02-18T01:25:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: cluster 2024-02-18T01:25:33.685157+0000 mgr.y (mgr.24475) 350 : cluster [DBG] pgmap v388: 292 pgs: 292 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.900398+0000 mon.c (mon.2) 312 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.900824+0000 mon.a (mon.0) 1588 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.902095+0000 mon.c (mon.2) 313 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.902466+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.905326+0000 mon.c (mon.2) 314 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.905726+0000 mon.a (mon.0) 1590 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.921079+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:33.921534+0000 mon.a (mon.0) 1591 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.878883+0000 mon.a (mon.0) 1592 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.879134+0000 mon.a (mon.0) 1593 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: cluster 2024-02-18T01:25:34.879324+0000 mon.a (mon.0) 1594 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.880532+0000 mon.c (mon.2) 315 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.880586+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:35.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.881140+0000 mon.a (mon.0) 1595 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[27669]: audit 2024-02-18T01:25:34.882062+0000 mon.a (mon.0) 1596 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: cluster 2024-02-18T01:25:33.685157+0000 mgr.y (mgr.24475) 350 : cluster [DBG] pgmap v388: 292 pgs: 292 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.900398+0000 mon.c (mon.2) 312 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.900824+0000 mon.a (mon.0) 1588 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.902095+0000 mon.c (mon.2) 313 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.902466+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.905326+0000 mon.c (mon.2) 314 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.905726+0000 mon.a (mon.0) 1590 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.921079+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:33.921534+0000 mon.a (mon.0) 1591 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.878883+0000 mon.a (mon.0) 1592 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.879134+0000 mon.a (mon.0) 1593 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: cluster 2024-02-18T01:25:34.879324+0000 mon.a (mon.0) 1594 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.880532+0000 mon.c (mon.2) 315 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.880586+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.881140+0000 mon.a (mon.0) 1595 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:34 smithi105 bash[19734]: audit 2024-02-18T01:25:34.882062+0000 mon.a (mon.0) 1596 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: cluster 2024-02-18T01:25:33.685157+0000 mgr.y (mgr.24475) 350 : cluster [DBG] pgmap v388: 292 pgs: 292 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.900398+0000 mon.c (mon.2) 312 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.900824+0000 mon.a (mon.0) 1588 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.902095+0000 mon.c (mon.2) 313 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.902466+0000 mon.a (mon.0) 1589 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.905326+0000 mon.c (mon.2) 314 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.905726+0000 mon.a (mon.0) 1590 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.921079+0000 mon.b (mon.1) 192 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:33.921534+0000 mon.a (mon.0) 1591 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.878883+0000 mon.a (mon.0) 1592 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi105-61649-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.879134+0000 mon.a (mon.0) 1593 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: cluster 2024-02-18T01:25:34.879324+0000 mon.a (mon.0) 1594 : cluster [DBG] osdmap e209: 8 total, 8 up, 8 in 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.880532+0000 mon.c (mon.2) 315 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.880586+0000 mon.b (mon.1) 193 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.881140+0000 mon.a (mon.0) 1595 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:35.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:34 smithi155 bash[20805]: audit 2024-02-18T01:25:34.882062+0000 mon.a (mon.0) 1596 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]: dispatch 2024-02-18T01:25:36.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:36 smithi105 bash[27669]: cluster 2024-02-18T01:25:35.873537+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:36.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:36 smithi105 bash[19734]: cluster 2024-02-18T01:25:35.873537+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:36.679 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:36] "GET /metrics HTTP/1.1" 200 188558 "" "Prometheus/2.33.4" 2024-02-18T01:25:36.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:36 smithi155 bash[20805]: cluster 2024-02-18T01:25:35.873537+0000 mon.a (mon.0) 1597 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: cluster 2024-02-18T01:25:35.686114+0000 mgr.y (mgr.24475) 351 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: audit 2024-02-18T01:25:36.319176+0000 mon.a (mon.0) 1598 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: cluster 2024-02-18T01:25:36.319504+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: audit 2024-02-18T01:25:36.320894+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.105:0/188025271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: audit 2024-02-18T01:25:36.326223+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: audit 2024-02-18T01:25:36.890414+0000 mon.a (mon.0) 1601 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: audit 2024-02-18T01:25:36.893012+0000 mon.a (mon.0) 1602 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:37.702 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[20805]: cluster 2024-02-18T01:25:36.897233+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-18T01:25:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: cluster 2024-02-18T01:25:35.686114+0000 mgr.y (mgr.24475) 351 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: audit 2024-02-18T01:25:36.319176+0000 mon.a (mon.0) 1598 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: cluster 2024-02-18T01:25:36.319504+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-18T01:25:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: audit 2024-02-18T01:25:36.320894+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.105:0/188025271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: audit 2024-02-18T01:25:36.326223+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: audit 2024-02-18T01:25:36.890414+0000 mon.a (mon.0) 1601 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: audit 2024-02-18T01:25:36.893012+0000 mon.a (mon.0) 1602 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[19734]: cluster 2024-02-18T01:25:36.897233+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: cluster 2024-02-18T01:25:35.686114+0000 mgr.y (mgr.24475) 351 : cluster [DBG] pgmap v391: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: audit 2024-02-18T01:25:36.319176+0000 mon.a (mon.0) 1598 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-25"}]': finished 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: cluster 2024-02-18T01:25:36.319504+0000 mon.a (mon.0) 1599 : cluster [DBG] osdmap e210: 8 total, 8 up, 8 in 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: audit 2024-02-18T01:25:36.320894+0000 mon.c (mon.2) 316 : audit [INF] from='client.? v1:172.21.15.105:0/188025271' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: audit 2024-02-18T01:25:36.326223+0000 mon.a (mon.0) 1600 : audit [INF] from='client.48074 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: audit 2024-02-18T01:25:36.890414+0000 mon.a (mon.0) 1601 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi105-61649-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:37.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: audit 2024-02-18T01:25:36.893012+0000 mon.a (mon.0) 1602 : audit [INF] from='client.48074 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi105-61657-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:37.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:37 smithi105 bash[27669]: cluster 2024-02-18T01:25:36.897233+0000 mon.a (mon.0) 1603 : cluster [DBG] osdmap e211: 8 total, 8 up, 8 in 2024-02-18T01:25:37.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[27669]: cluster 2024-02-18T01:25:37.688241+0000 mgr.y (mgr.24475) 352 : cluster [DBG] pgmap v394: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[27669]: audit 2024-02-18T01:25:37.699669+0000 mgr.y (mgr.24475) 353 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[27669]: cluster 2024-02-18T01:25:37.893694+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[27669]: audit 2024-02-18T01:25:37.900801+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[27669]: audit 2024-02-18T01:25:37.901629+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[19734]: cluster 2024-02-18T01:25:37.688241+0000 mgr.y (mgr.24475) 352 : cluster [DBG] pgmap v394: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[19734]: audit 2024-02-18T01:25:37.699669+0000 mgr.y (mgr.24475) 353 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[19734]: cluster 2024-02-18T01:25:37.893694+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[19734]: audit 2024-02-18T01:25:37.900801+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:38 smithi105 bash[19734]: audit 2024-02-18T01:25:37.901629+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:38 smithi155 bash[20805]: cluster 2024-02-18T01:25:37.688241+0000 mgr.y (mgr.24475) 352 : cluster [DBG] pgmap v394: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:38 smithi155 bash[20805]: audit 2024-02-18T01:25:37.699669+0000 mgr.y (mgr.24475) 353 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:38 smithi155 bash[20805]: cluster 2024-02-18T01:25:37.893694+0000 mon.a (mon.0) 1604 : cluster [DBG] osdmap e212: 8 total, 8 up, 8 in 2024-02-18T01:25:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:38 smithi155 bash[20805]: audit 2024-02-18T01:25:37.900801+0000 mon.b (mon.1) 194 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:38 smithi155 bash[20805]: audit 2024-02-18T01:25:37.901629+0000 mon.a (mon.0) 1605 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:38.897528+0000 mon.a (mon.0) 1606 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: cluster 2024-02-18T01:25:38.897859+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:38.899458+0000 mon.c (mon.2) 317 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:38.900316+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.105:0/2689686242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:38.900913+0000 mon.a (mon.0) 1608 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:38.902576+0000 mon.a (mon.0) 1609 : audit [INF] from='client.48817 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:39.897063+0000 mon.a (mon.0) 1610 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:39.897210+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48817 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: cluster 2024-02-18T01:25:39.897464+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:39.902354+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[19734]: audit 2024-02-18T01:25:39.902886+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:38.897528+0000 mon.a (mon.0) 1606 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: cluster 2024-02-18T01:25:38.897859+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:38.899458+0000 mon.c (mon.2) 317 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:38.900316+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.105:0/2689686242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:38.900913+0000 mon.a (mon.0) 1608 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:38.902576+0000 mon.a (mon.0) 1609 : audit [INF] from='client.48817 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:39.897063+0000 mon.a (mon.0) 1610 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:39.897210+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48817 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: cluster 2024-02-18T01:25:39.897464+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:39.902354+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:39 smithi105 bash[27669]: audit 2024-02-18T01:25:39.902886+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:38.897528+0000 mon.a (mon.0) 1606 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: cluster 2024-02-18T01:25:38.897859+0000 mon.a (mon.0) 1607 : cluster [DBG] osdmap e213: 8 total, 8 up, 8 in 2024-02-18T01:25:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:38.899458+0000 mon.c (mon.2) 317 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:38.900316+0000 mon.b (mon.1) 195 : audit [INF] from='client.? v1:172.21.15.105:0/2689686242' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:38.900913+0000 mon.a (mon.0) 1608 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:38.902576+0000 mon.a (mon.0) 1609 : audit [INF] from='client.48817 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:39.897063+0000 mon.a (mon.0) 1610 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:39.897210+0000 mon.a (mon.0) 1611 : audit [INF] from='client.48817 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi105-61657-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: cluster 2024-02-18T01:25:39.897464+0000 mon.a (mon.0) 1612 : cluster [DBG] osdmap e214: 8 total, 8 up, 8 in 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:39.902354+0000 mon.c (mon.2) 318 : audit [INF] from='client.? v1:172.21.15.105:0/1369468789' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:39 smithi155 bash[20805]: audit 2024-02-18T01:25:39.902886+0000 mon.a (mon.0) 1613 : audit [INF] from='client.48071 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]: dispatch 2024-02-18T01:25:40.926 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: cluster 2024-02-18T01:25:39.689141+0000 mgr.y (mgr.24475) 354 : cluster [DBG] pgmap v397: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:40.926 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:39.931622+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:40.926 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:39.931819+0000 mon.a (mon.0) 1614 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:40.926 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:40.901274+0000 mon.a (mon.0) 1615 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:40.926 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:40.901550+0000 mon.a (mon.0) 1616 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: cluster 2024-02-18T01:25:39.689141+0000 mgr.y (mgr.24475) 354 : cluster [DBG] pgmap v397: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:39.931622+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:39.931819+0000 mon.a (mon.0) 1614 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:40.901274+0000 mon.a (mon.0) 1615 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:40.901550+0000 mon.a (mon.0) 1616 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: cluster 2024-02-18T01:25:40.901704+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:40.904627+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:40.927 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[27669]: audit 2024-02-18T01:25:40.906163+0000 mon.a (mon.0) 1618 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: cluster 2024-02-18T01:25:40.901704+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-18T01:25:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:40.904627+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:40 smithi105 bash[19734]: audit 2024-02-18T01:25:40.906163+0000 mon.a (mon.0) 1618 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: cluster 2024-02-18T01:25:39.689141+0000 mgr.y (mgr.24475) 354 : cluster [DBG] pgmap v397: 260 pgs: 64 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:39.931622+0000 mon.b (mon.1) 196 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:39.931819+0000 mon.a (mon.0) 1614 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:40.901274+0000 mon.a (mon.0) 1615 : audit [INF] from='client.48071 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi105-61649-35"}]': finished 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:40.901550+0000 mon.a (mon.0) 1616 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: cluster 2024-02-18T01:25:40.901704+0000 mon.a (mon.0) 1617 : cluster [DBG] osdmap e215: 8 total, 8 up, 8 in 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:40.904627+0000 mon.b (mon.1) 197 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:40 smithi155 bash[20805]: audit 2024-02-18T01:25:40.906163+0000 mon.a (mon.0) 1618 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.929126+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.930768+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.931194+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.931781+0000 mon.c (mon.2) 319 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.932145+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.933124+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.933426+0000 mon.a (mon.0) 1623 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.933943+0000 mon.c (mon.2) 321 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:40.934211+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.902858+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.903321+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.903489+0000 mon.a (mon.0) 1627 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: cluster 2024-02-18T01:25:41.903649+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.904690+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.929126+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.930768+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.931194+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.931781+0000 mon.c (mon.2) 319 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.932145+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.933124+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.933426+0000 mon.a (mon.0) 1623 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.933943+0000 mon.c (mon.2) 321 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:40.934211+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.902858+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.903321+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.903489+0000 mon.a (mon.0) 1627 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: cluster 2024-02-18T01:25:41.903649+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.904690+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.904773+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.904934+0000 mon.c (mon.2) 322 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.905424+0000 mon.a (mon.0) 1630 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[19734]: audit 2024-02-18T01:25:41.906172+0000 mon.a (mon.0) 1631 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.904773+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.904934+0000 mon.c (mon.2) 322 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.905424+0000 mon.a (mon.0) 1630 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:41 smithi105 bash[27669]: audit 2024-02-18T01:25:41.906172+0000 mon.a (mon.0) 1631 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.245 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.929126+0000 mon.a (mon.0) 1619 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.930768+0000 mon.a (mon.0) 1620 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.931194+0000 mon.a (mon.0) 1621 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.931781+0000 mon.c (mon.2) 319 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.932145+0000 mon.a (mon.0) 1622 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.933124+0000 mon.c (mon.2) 320 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.933426+0000 mon.a (mon.0) 1623 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.933943+0000 mon.c (mon.2) 321 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:40.934211+0000 mon.a (mon.0) 1624 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.902858+0000 mon.a (mon.0) 1625 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.903321+0000 mon.a (mon.0) 1626 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.903489+0000 mon.a (mon.0) 1627 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-45", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: cluster 2024-02-18T01:25:41.903649+0000 mon.a (mon.0) 1628 : cluster [DBG] osdmap e216: 8 total, 8 up, 8 in 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.904690+0000 mon.b (mon.1) 198 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.904773+0000 mon.a (mon.0) 1629 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:42.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.904934+0000 mon.c (mon.2) 322 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:42.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.905424+0000 mon.a (mon.0) 1630 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]: dispatch 2024-02-18T01:25:42.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:41 smithi155 bash[20805]: audit 2024-02-18T01:25:41.906172+0000 mon.a (mon.0) 1631 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:44.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:43 smithi105 bash[27669]: cluster 2024-02-18T01:25:41.689963+0000 mgr.y (mgr.24475) 355 : cluster [DBG] pgmap v400: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:44.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:43 smithi105 bash[27669]: cluster 2024-02-18T01:25:42.897624+0000 mon.a (mon.0) 1632 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:44.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:43 smithi105 bash[19734]: cluster 2024-02-18T01:25:41.689963+0000 mgr.y (mgr.24475) 355 : cluster [DBG] pgmap v400: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:44.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:43 smithi105 bash[19734]: cluster 2024-02-18T01:25:42.897624+0000 mon.a (mon.0) 1632 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:44.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:43 smithi155 bash[20805]: cluster 2024-02-18T01:25:41.689963+0000 mgr.y (mgr.24475) 355 : cluster [DBG] pgmap v400: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:44.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:43 smithi155 bash[20805]: cluster 2024-02-18T01:25:42.897624+0000 mon.a (mon.0) 1632 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: audit 2024-02-18T01:25:43.332635+0000 mon.a (mon.0) 1633 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]': finished 2024-02-18T01:25:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: cluster 2024-02-18T01:25:43.690844+0000 mgr.y (mgr.24475) 356 : cluster [DBG] pgmap v403: 228 pgs: 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: cluster 2024-02-18T01:25:43.740412+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-18T01:25:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: audit 2024-02-18T01:25:43.917904+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: audit 2024-02-18T01:25:43.918057+0000 mon.a (mon.0) 1636 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: cluster 2024-02-18T01:25:43.918516+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: audit 2024-02-18T01:25:43.951403+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[27669]: audit 2024-02-18T01:25:43.951598+0000 mon.a (mon.0) 1638 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: audit 2024-02-18T01:25:43.332635+0000 mon.a (mon.0) 1633 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]': finished 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: cluster 2024-02-18T01:25:43.690844+0000 mgr.y (mgr.24475) 356 : cluster [DBG] pgmap v403: 228 pgs: 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: cluster 2024-02-18T01:25:43.740412+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: audit 2024-02-18T01:25:43.917904+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: audit 2024-02-18T01:25:43.918057+0000 mon.a (mon.0) 1636 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:45.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: cluster 2024-02-18T01:25:43.918516+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-18T01:25:45.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: audit 2024-02-18T01:25:43.951403+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:45.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:44 smithi105 bash[19734]: audit 2024-02-18T01:25:43.951598+0000 mon.a (mon.0) 1638 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: audit 2024-02-18T01:25:43.332635+0000 mon.a (mon.0) 1633 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-27", "mode": "writeback"}]': finished 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: cluster 2024-02-18T01:25:43.690844+0000 mgr.y (mgr.24475) 356 : cluster [DBG] pgmap v403: 228 pgs: 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: cluster 2024-02-18T01:25:43.740412+0000 mon.a (mon.0) 1634 : cluster [DBG] osdmap e217: 8 total, 8 up, 8 in 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: audit 2024-02-18T01:25:43.917904+0000 mon.a (mon.0) 1635 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi105-61649-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: audit 2024-02-18T01:25:43.918057+0000 mon.a (mon.0) 1636 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-45", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: cluster 2024-02-18T01:25:43.918516+0000 mon.a (mon.0) 1637 : cluster [DBG] osdmap e218: 8 total, 8 up, 8 in 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: audit 2024-02-18T01:25:43.951403+0000 mon.b (mon.1) 199 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:45.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:44 smithi155 bash[20805]: audit 2024-02-18T01:25:43.951598+0000 mon.a (mon.0) 1638 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:46.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[19734]: audit 2024-02-18T01:25:44.924194+0000 mon.a (mon.0) 1639 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:46.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[19734]: cluster 2024-02-18T01:25:44.925089+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-18T01:25:46.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[19734]: audit 2024-02-18T01:25:44.926256+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[19734]: audit 2024-02-18T01:25:44.926796+0000 mon.a (mon.0) 1641 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[27669]: audit 2024-02-18T01:25:44.924194+0000 mon.a (mon.0) 1639 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[27669]: cluster 2024-02-18T01:25:44.925089+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-18T01:25:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[27669]: audit 2024-02-18T01:25:44.926256+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:45 smithi105 bash[27669]: audit 2024-02-18T01:25:44.926796+0000 mon.a (mon.0) 1641 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:45 smithi155 bash[20805]: audit 2024-02-18T01:25:44.924194+0000 mon.a (mon.0) 1639 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:45 smithi155 bash[20805]: cluster 2024-02-18T01:25:44.925089+0000 mon.a (mon.0) 1640 : cluster [DBG] osdmap e219: 8 total, 8 up, 8 in 2024-02-18T01:25:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:45 smithi155 bash[20805]: audit 2024-02-18T01:25:44.926256+0000 mon.b (mon.1) 200 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:45 smithi155 bash[20805]: audit 2024-02-18T01:25:44.926796+0000 mon.a (mon.0) 1641 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]: dispatch 2024-02-18T01:25:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:46] "GET /metrics HTTP/1.1" 200 188558 "" "Prometheus/2.33.4" 2024-02-18T01:25:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: cluster 2024-02-18T01:25:45.691750+0000 mgr.y (mgr.24475) 357 : cluster [DBG] pgmap v406: 244 pgs: 16 unknown, 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: cluster 2024-02-18T01:25:45.916133+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: audit 2024-02-18T01:25:45.931940+0000 mon.a (mon.0) 1643 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: cluster 2024-02-18T01:25:45.932209+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: audit 2024-02-18T01:25:45.934066+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: audit 2024-02-18T01:25:45.934440+0000 mon.c (mon.2) 323 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[19734]: audit 2024-02-18T01:25:45.934921+0000 mon.a (mon.0) 1646 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: cluster 2024-02-18T01:25:45.691750+0000 mgr.y (mgr.24475) 357 : cluster [DBG] pgmap v406: 244 pgs: 16 unknown, 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: cluster 2024-02-18T01:25:45.916133+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: audit 2024-02-18T01:25:45.931940+0000 mon.a (mon.0) 1643 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: cluster 2024-02-18T01:25:45.932209+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-18T01:25:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: audit 2024-02-18T01:25:45.934066+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: audit 2024-02-18T01:25:45.934440+0000 mon.c (mon.2) 323 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:46 smithi105 bash[27669]: audit 2024-02-18T01:25:45.934921+0000 mon.a (mon.0) 1646 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: cluster 2024-02-18T01:25:45.691750+0000 mgr.y (mgr.24475) 357 : cluster [DBG] pgmap v406: 244 pgs: 16 unknown, 228 active+clean; 6.9 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: cluster 2024-02-18T01:25:45.916133+0000 mon.a (mon.0) 1642 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: audit 2024-02-18T01:25:45.931940+0000 mon.a (mon.0) 1643 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-27"}]': finished 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: cluster 2024-02-18T01:25:45.932209+0000 mon.a (mon.0) 1644 : cluster [DBG] osdmap e220: 8 total, 8 up, 8 in 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: audit 2024-02-18T01:25:45.934066+0000 mon.a (mon.0) 1645 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: audit 2024-02-18T01:25:45.934440+0000 mon.c (mon.2) 323 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:47.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:46 smithi155 bash[20805]: audit 2024-02-18T01:25:45.934921+0000 mon.a (mon.0) 1646 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:47.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: audit 2024-02-18T01:25:46.937471+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: audit 2024-02-18T01:25:46.937602+0000 mon.a (mon.0) 1648 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:47.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: cluster 2024-02-18T01:25:46.937761+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-18T01:25:47.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: audit 2024-02-18T01:25:46.938666+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:47.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: audit 2024-02-18T01:25:46.939261+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:47.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:47 smithi155 bash[20805]: audit 2024-02-18T01:25:46.939650+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: audit 2024-02-18T01:25:46.937471+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: audit 2024-02-18T01:25:46.937602+0000 mon.a (mon.0) 1648 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: cluster 2024-02-18T01:25:46.937761+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: audit 2024-02-18T01:25:46.938666+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: audit 2024-02-18T01:25:46.939261+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[19734]: audit 2024-02-18T01:25:46.939650+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: audit 2024-02-18T01:25:46.937471+0000 mon.a (mon.0) 1647 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: audit 2024-02-18T01:25:46.937602+0000 mon.a (mon.0) 1648 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: cluster 2024-02-18T01:25:46.937761+0000 mon.a (mon.0) 1649 : cluster [DBG] osdmap e221: 8 total, 8 up, 8 in 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: audit 2024-02-18T01:25:46.938666+0000 mon.c (mon.2) 324 : audit [INF] from='client.? v1:172.21.15.105:0/1122773445' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: audit 2024-02-18T01:25:46.939261+0000 mon.a (mon.0) 1650 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]: dispatch 2024-02-18T01:25:48.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:47 smithi105 bash[27669]: audit 2024-02-18T01:25:46.939650+0000 mon.a (mon.0) 1651 : audit [INF] from='client.48089 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]: dispatch 2024-02-18T01:25:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: cluster 2024-02-18T01:25:47.693763+0000 mgr.y (mgr.24475) 358 : cluster [DBG] pgmap v409: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.710799+0000 mgr.y (mgr.24475) 359 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.945501+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.946963+0000 mon.a (mon.0) 1653 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: cluster 2024-02-18T01:25:47.947110+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.948778+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.956519+0000 mon.a (mon.0) 1655 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.971297+0000 mon.c (mon.2) 325 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.971678+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.972828+0000 mon.c (mon.2) 326 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.973137+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.974052+0000 mon.c (mon.2) 327 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.974582+0000 mon.a (mon.0) 1658 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.975195+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.975286+0000 mon.a (mon.0) 1659 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: cluster 2024-02-18T01:25:47.693763+0000 mgr.y (mgr.24475) 358 : cluster [DBG] pgmap v409: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.710799+0000 mgr.y (mgr.24475) 359 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.945501+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.946963+0000 mon.a (mon.0) 1653 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: cluster 2024-02-18T01:25:47.947110+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.948778+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.956519+0000 mon.a (mon.0) 1655 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.971297+0000 mon.c (mon.2) 325 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.971678+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.972828+0000 mon.c (mon.2) 326 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.973137+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.974052+0000 mon.c (mon.2) 327 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.974582+0000 mon.a (mon.0) 1658 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.975195+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.975286+0000 mon.a (mon.0) 1659 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.976689+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.976750+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.977489+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:47.977603+0000 mon.a (mon.0) 1661 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.867936+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.868077+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.868170+0000 mon.a (mon.0) 1664 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: cluster 2024-02-18T01:25:48.868251+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.873038+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.873974+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.874402+0000 mon.a (mon.0) 1666 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.875880+0000 mon.a (mon.0) 1667 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.875962+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[19734]: audit 2024-02-18T01:25:48.876826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.976689+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.976750+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.977489+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:47.977603+0000 mon.a (mon.0) 1661 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.867936+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.868077+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.868170+0000 mon.a (mon.0) 1664 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: cluster 2024-02-18T01:25:48.868251+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-18T01:25:49.245 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.873038+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.873974+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.874402+0000 mon.a (mon.0) 1666 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.875880+0000 mon.a (mon.0) 1667 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.875962+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:48 smithi105 bash[27669]: audit 2024-02-18T01:25:48.876826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: cluster 2024-02-18T01:25:47.693763+0000 mgr.y (mgr.24475) 358 : cluster [DBG] pgmap v409: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:49.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.710799+0000 mgr.y (mgr.24475) 359 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.945501+0000 mon.a (mon.0) 1652 : audit [INF] from='client.? v1:172.21.15.105:0/4114229627' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi105-61649-36"}]': finished 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.946963+0000 mon.a (mon.0) 1653 : audit [INF] from='client.48089 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-45"}]': finished 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: cluster 2024-02-18T01:25:47.947110+0000 mon.a (mon.0) 1654 : cluster [DBG] osdmap e222: 8 total, 8 up, 8 in 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.948778+0000 mon.b (mon.1) 201 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.956519+0000 mon.a (mon.0) 1655 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.971297+0000 mon.c (mon.2) 325 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.971678+0000 mon.a (mon.0) 1656 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.972828+0000 mon.c (mon.2) 326 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.973137+0000 mon.a (mon.0) 1657 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.974052+0000 mon.c (mon.2) 327 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.974582+0000 mon.a (mon.0) 1658 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.975195+0000 mon.b (mon.1) 202 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.975286+0000 mon.a (mon.0) 1659 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.976689+0000 mon.b (mon.1) 203 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.976750+0000 mon.a (mon.0) 1660 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.977489+0000 mon.b (mon.1) 204 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:47.977603+0000 mon.a (mon.0) 1661 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.867936+0000 mon.a (mon.0) 1662 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.868077+0000 mon.a (mon.0) 1663 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi105-61657-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.868170+0000 mon.a (mon.0) 1664 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi105-61649-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: cluster 2024-02-18T01:25:48.868251+0000 mon.a (mon.0) 1665 : cluster [DBG] osdmap e223: 8 total, 8 up, 8 in 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.873038+0000 mon.c (mon.2) 328 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.248 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.873974+0000 mon.b (mon.1) 205 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.874402+0000 mon.a (mon.0) 1666 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:49.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.875880+0000 mon.a (mon.0) 1667 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:49.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.875962+0000 mon.b (mon.1) 206 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:49.249 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:48 smithi155 bash[20805]: audit 2024-02-18T01:25:48.876826+0000 mon.a (mon.0) 1668 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[19734]: cluster 2024-02-18T01:25:49.694623+0000 mgr.y (mgr.24475) 360 : cluster [DBG] pgmap v412: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[19734]: audit 2024-02-18T01:25:50.726148+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[19734]: cluster 2024-02-18T01:25:50.726354+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[19734]: audit 2024-02-18T01:25:50.727868+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[19734]: audit 2024-02-18T01:25:50.731075+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[27669]: cluster 2024-02-18T01:25:49.694623+0000 mgr.y (mgr.24475) 360 : cluster [DBG] pgmap v412: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[27669]: audit 2024-02-18T01:25:50.726148+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[27669]: cluster 2024-02-18T01:25:50.726354+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[27669]: audit 2024-02-18T01:25:50.727868+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:50 smithi105 bash[27669]: audit 2024-02-18T01:25:50.731075+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:50 smithi155 bash[20805]: cluster 2024-02-18T01:25:49.694623+0000 mgr.y (mgr.24475) 360 : cluster [DBG] pgmap v412: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:25:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:50 smithi155 bash[20805]: audit 2024-02-18T01:25:50.726148+0000 mon.a (mon.0) 1669 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:50 smithi155 bash[20805]: cluster 2024-02-18T01:25:50.726354+0000 mon.a (mon.0) 1670 : cluster [DBG] osdmap e224: 8 total, 8 up, 8 in 2024-02-18T01:25:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:50 smithi155 bash[20805]: audit 2024-02-18T01:25:50.727868+0000 mon.b (mon.1) 207 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:50 smithi155 bash[20805]: audit 2024-02-18T01:25:50.731075+0000 mon.a (mon.0) 1671 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:50.882556+0000 mon.a (mon.0) 1672 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:52.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:50.882788+0000 mon.a (mon.0) 1673 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:50.882897+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: cluster 2024-02-18T01:25:50.882971+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:50.885172+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:50.892332+0000 mon.a (mon.0) 1676 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: cluster 2024-02-18T01:25:51.870867+0000 mon.a (mon.0) 1677 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: audit 2024-02-18T01:25:51.881498+0000 mon.a (mon.0) 1678 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]': finished 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[19734]: cluster 2024-02-18T01:25:51.881783+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:50.882556+0000 mon.a (mon.0) 1672 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:50.882788+0000 mon.a (mon.0) 1673 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:52.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:50.882897+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: cluster 2024-02-18T01:25:50.882971+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:50.885172+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:50.892332+0000 mon.a (mon.0) 1676 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: cluster 2024-02-18T01:25:51.870867+0000 mon.a (mon.0) 1677 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: audit 2024-02-18T01:25:51.881498+0000 mon.a (mon.0) 1678 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]': finished 2024-02-18T01:25:52.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:51 smithi105 bash[27669]: cluster 2024-02-18T01:25:51.881783+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:50.882556+0000 mon.a (mon.0) 1672 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi105-61657-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:50.882788+0000 mon.a (mon.0) 1673 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi105-61649-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:50.882897+0000 mon.a (mon.0) 1674 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: cluster 2024-02-18T01:25:50.882971+0000 mon.a (mon.0) 1675 : cluster [DBG] osdmap e225: 8 total, 8 up, 8 in 2024-02-18T01:25:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:50.885172+0000 mon.b (mon.1) 208 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:50.892332+0000 mon.a (mon.0) 1676 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]: dispatch 2024-02-18T01:25:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: cluster 2024-02-18T01:25:51.870867+0000 mon.a (mon.0) 1677 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:25:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: audit 2024-02-18T01:25:51.881498+0000 mon.a (mon.0) 1678 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-29", "mode": "writeback"}]': finished 2024-02-18T01:25:52.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:51 smithi155 bash[20805]: cluster 2024-02-18T01:25:51.881783+0000 mon.a (mon.0) 1679 : cluster [DBG] osdmap e226: 8 total, 8 up, 8 in 2024-02-18T01:25:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[19734]: cluster 2024-02-18T01:25:51.695568+0000 mgr.y (mgr.24475) 361 : cluster [DBG] pgmap v415: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[19734]: audit 2024-02-18T01:25:51.975438+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[19734]: audit 2024-02-18T01:25:51.975951+0000 mon.a (mon.0) 1680 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[27669]: cluster 2024-02-18T01:25:51.695568+0000 mgr.y (mgr.24475) 361 : cluster [DBG] pgmap v415: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[27669]: audit 2024-02-18T01:25:51.975438+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:53.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:52 smithi105 bash[27669]: audit 2024-02-18T01:25:51.975951+0000 mon.a (mon.0) 1680 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:52 smithi155 bash[20805]: cluster 2024-02-18T01:25:51.695568+0000 mgr.y (mgr.24475) 361 : cluster [DBG] pgmap v415: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:25:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:52 smithi155 bash[20805]: audit 2024-02-18T01:25:51.975438+0000 mon.b (mon.1) 209 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:53.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:52 smithi155 bash[20805]: audit 2024-02-18T01:25:51.975951+0000 mon.a (mon.0) 1680 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:25:54.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.905155+0000 mon.a (mon.0) 1681 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: cluster 2024-02-18T01:25:52.905367+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.906725+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.907099+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.907414+0000 mon.c (mon.2) 329 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.911646+0000 mon.a (mon.0) 1683 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.911979+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:52.916940+0000 mon.a (mon.0) 1685 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: cluster 2024-02-18T01:25:53.896574+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.906239+0000 mon.a (mon.0) 1687 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.906494+0000 mon.a (mon.0) 1688 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.906998+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:54.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: cluster 2024-02-18T01:25:53.907164+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.907983+0000 mon.c (mon.2) 330 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.908242+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.908765+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[19734]: audit 2024-02-18T01:25:53.909119+0000 mon.a (mon.0) 1692 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.905155+0000 mon.a (mon.0) 1681 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: cluster 2024-02-18T01:25:52.905367+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.906725+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.907099+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.907414+0000 mon.c (mon.2) 329 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.911646+0000 mon.a (mon.0) 1683 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.911979+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:52.916940+0000 mon.a (mon.0) 1685 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: cluster 2024-02-18T01:25:53.896574+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.906239+0000 mon.a (mon.0) 1687 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.906494+0000 mon.a (mon.0) 1688 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.906998+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: cluster 2024-02-18T01:25:53.907164+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.907983+0000 mon.c (mon.2) 330 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.908242+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.908765+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:53 smithi105 bash[27669]: audit 2024-02-18T01:25:53.909119+0000 mon.a (mon.0) 1692 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.905155+0000 mon.a (mon.0) 1681 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: cluster 2024-02-18T01:25:52.905367+0000 mon.a (mon.0) 1682 : cluster [DBG] osdmap e227: 8 total, 8 up, 8 in 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.906725+0000 mon.b (mon.1) 210 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.907099+0000 mon.b (mon.1) 211 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.907414+0000 mon.c (mon.2) 329 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.911646+0000 mon.a (mon.0) 1683 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.911979+0000 mon.a (mon.0) 1684 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:52.916940+0000 mon.a (mon.0) 1685 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: cluster 2024-02-18T01:25:53.896574+0000 mon.a (mon.0) 1686 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.906239+0000 mon.a (mon.0) 1687 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-29"}]': finished 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.906494+0000 mon.a (mon.0) 1688 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.906998+0000 mon.a (mon.0) 1689 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: cluster 2024-02-18T01:25:53.907164+0000 mon.a (mon.0) 1690 : cluster [DBG] osdmap e228: 8 total, 8 up, 8 in 2024-02-18T01:25:54.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.907983+0000 mon.c (mon.2) 330 : audit [INF] from='client.? v1:172.21.15.105:0/1095363833' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.908242+0000 mon.b (mon.1) 212 : audit [INF] from='client.? v1:172.21.15.105:0/2129220749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.908765+0000 mon.a (mon.0) 1691 : audit [INF] from='client.48095 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]: dispatch 2024-02-18T01:25:54.246 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:53 smithi155 bash[20805]: audit 2024-02-18T01:25:53.909119+0000 mon.a (mon.0) 1692 : audit [INF] from='client.48838 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]: dispatch 2024-02-18T01:25:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[19734]: cluster 2024-02-18T01:25:53.696447+0000 mgr.y (mgr.24475) 362 : cluster [DBG] pgmap v418: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[19734]: audit 2024-02-18T01:25:54.908756+0000 mon.a (mon.0) 1693 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[19734]: audit 2024-02-18T01:25:54.909003+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:55.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[19734]: cluster 2024-02-18T01:25:54.909262+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-18T01:25:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[27669]: cluster 2024-02-18T01:25:53.696447+0000 mgr.y (mgr.24475) 362 : cluster [DBG] pgmap v418: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[27669]: audit 2024-02-18T01:25:54.908756+0000 mon.a (mon.0) 1693 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[27669]: audit 2024-02-18T01:25:54.909003+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:55 smithi105 bash[27669]: cluster 2024-02-18T01:25:54.909262+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-18T01:25:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:55 smithi155 bash[20805]: cluster 2024-02-18T01:25:53.696447+0000 mgr.y (mgr.24475) 362 : cluster [DBG] pgmap v418: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:25:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:55 smithi155 bash[20805]: audit 2024-02-18T01:25:54.908756+0000 mon.a (mon.0) 1693 : audit [INF] from='client.48095 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi105-61657-46"}]': finished 2024-02-18T01:25:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:55 smithi155 bash[20805]: audit 2024-02-18T01:25:54.909003+0000 mon.a (mon.0) 1694 : audit [INF] from='client.48838 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi105-61649-37"}]': finished 2024-02-18T01:25:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:55 smithi155 bash[20805]: cluster 2024-02-18T01:25:54.909262+0000 mon.a (mon.0) 1695 : cluster [DBG] osdmap e229: 8 total, 8 up, 8 in 2024-02-18T01:25:56.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:54.932976+0000 mon.c (mon.2) 331 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:54.934596+0000 mon.c (mon.2) 332 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.350462+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.350772+0000 mon.a (mon.0) 1697 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.351569+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.357191+0000 mon.c (mon.2) 334 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.357479+0000 mon.a (mon.0) 1698 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.358436+0000 mon.c (mon.2) 335 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.358544+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.359338+0000 mon.a (mon.0) 1700 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.359426+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.360243+0000 mon.a (mon.0) 1701 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: cluster 2024-02-18T01:25:55.697287+0000 mgr.y (mgr.24475) 363 : cluster [DBG] pgmap v421: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.911979+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:54.932976+0000 mon.c (mon.2) 331 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:54.934596+0000 mon.c (mon.2) 332 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.350462+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.350772+0000 mon.a (mon.0) 1697 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.351569+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.357191+0000 mon.c (mon.2) 334 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.357479+0000 mon.a (mon.0) 1698 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.358436+0000 mon.c (mon.2) 335 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.358544+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.359338+0000 mon.a (mon.0) 1700 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.359426+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.360243+0000 mon.a (mon.0) 1701 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: cluster 2024-02-18T01:25:55.697287+0000 mgr.y (mgr.24475) 363 : cluster [DBG] pgmap v421: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.911979+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.912174+0000 mon.a (mon.0) 1703 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: cluster 2024-02-18T01:25:55.912807+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.913940+0000 mon.c (mon.2) 337 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.914243+0000 mon.c (mon.2) 338 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.915787+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.916155+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.916587+0000 mon.a (mon.0) 1706 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[27669]: audit 2024-02-18T01:25:55.917282+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.912174+0000 mon.a (mon.0) 1703 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: cluster 2024-02-18T01:25:55.912807+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.913940+0000 mon.c (mon.2) 337 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.914243+0000 mon.c (mon.2) 338 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.915787+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.916155+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.916587+0000 mon.a (mon.0) 1706 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[19734]: audit 2024-02-18T01:25:55.917282+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:54.932976+0000 mon.c (mon.2) 331 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:54.934596+0000 mon.c (mon.2) 332 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.350462+0000 mon.a (mon.0) 1696 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.350772+0000 mon.a (mon.0) 1697 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.351569+0000 mon.c (mon.2) 333 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.357191+0000 mon.c (mon.2) 334 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.357479+0000 mon.a (mon.0) 1698 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.358436+0000 mon.c (mon.2) 335 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.358544+0000 mon.a (mon.0) 1699 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.359338+0000 mon.a (mon.0) 1700 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.359426+0000 mon.c (mon.2) 336 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.360243+0000 mon.a (mon.0) 1701 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: cluster 2024-02-18T01:25:55.697287+0000 mgr.y (mgr.24475) 363 : cluster [DBG] pgmap v421: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.911979+0000 mon.a (mon.0) 1702 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi105-61649-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.912174+0000 mon.a (mon.0) 1703 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi105-61657-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: cluster 2024-02-18T01:25:55.912807+0000 mon.a (mon.0) 1704 : cluster [DBG] osdmap e230: 8 total, 8 up, 8 in 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.913940+0000 mon.c (mon.2) 337 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.914243+0000 mon.c (mon.2) 338 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.915787+0000 mon.b (mon.1) 213 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.916155+0000 mon.a (mon.0) 1705 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.916587+0000 mon.a (mon.0) 1706 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:25:56.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:56 smithi155 bash[20805]: audit 2024-02-18T01:25:55.917282+0000 mon.a (mon.0) 1707 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:25:56.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:25:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:25:56] "GET /metrics HTTP/1.1" 200 191318 "" "Prometheus/2.33.4" 2024-02-18T01:25:57.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:25:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:25:57.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:57 smithi155 bash[20805]: audit 2024-02-18T01:25:56.916569+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:57.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:57 smithi155 bash[20805]: audit 2024-02-18T01:25:56.923180+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:57.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:57 smithi155 bash[20805]: cluster 2024-02-18T01:25:57.746136+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-18T01:25:57.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:57 smithi155 bash[20805]: audit 2024-02-18T01:25:57.748178+0000 mon.a (mon.0) 1710 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[19734]: audit 2024-02-18T01:25:56.916569+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[19734]: audit 2024-02-18T01:25:56.923180+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[19734]: cluster 2024-02-18T01:25:57.746136+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-18T01:25:58.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[19734]: audit 2024-02-18T01:25:57.748178+0000 mon.a (mon.0) 1710 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[27669]: audit 2024-02-18T01:25:56.916569+0000 mon.a (mon.0) 1708 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:25:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[27669]: audit 2024-02-18T01:25:56.923180+0000 mon.b (mon.1) 214 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[27669]: cluster 2024-02-18T01:25:57.746136+0000 mon.a (mon.0) 1709 : cluster [DBG] osdmap e231: 8 total, 8 up, 8 in 2024-02-18T01:25:58.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:57 smithi105 bash[27669]: audit 2024-02-18T01:25:57.748178+0000 mon.a (mon.0) 1710 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:25:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: cluster 2024-02-18T01:25:57.699390+0000 mgr.y (mgr.24475) 364 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:25:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.717832+0000 mgr.y (mgr.24475) 365 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.923773+0000 mon.a (mon.0) 1711 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:25:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.923931+0000 mon.a (mon.0) 1712 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.924080+0000 mon.a (mon.0) 1713 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: cluster 2024-02-18T01:25:57.924204+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.929052+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[19734]: audit 2024-02-18T01:25:57.931645+0000 mon.a (mon.0) 1715 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: cluster 2024-02-18T01:25:57.699390+0000 mgr.y (mgr.24475) 364 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.717832+0000 mgr.y (mgr.24475) 365 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.923773+0000 mon.a (mon.0) 1711 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.923931+0000 mon.a (mon.0) 1712 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.924080+0000 mon.a (mon.0) 1713 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: cluster 2024-02-18T01:25:57.924204+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-18T01:25:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.929052+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:25:59.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:58 smithi105 bash[27669]: audit 2024-02-18T01:25:57.931645+0000 mon.a (mon.0) 1715 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: cluster 2024-02-18T01:25:57.699390+0000 mgr.y (mgr.24475) 364 : cluster [DBG] pgmap v424: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.717832+0000 mgr.y (mgr.24475) 365 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.923773+0000 mon.a (mon.0) 1711 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi105-61649-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.923931+0000 mon.a (mon.0) 1712 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi105-61657-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.924080+0000 mon.a (mon.0) 1713 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: cluster 2024-02-18T01:25:57.924204+0000 mon.a (mon.0) 1714 : cluster [DBG] osdmap e232: 8 total, 8 up, 8 in 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.929052+0000 mon.b (mon.1) 215 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:25:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:58 smithi155 bash[20805]: audit 2024-02-18T01:25:57.931645+0000 mon.a (mon.0) 1715 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:00.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[27669]: audit 2024-02-18T01:25:58.935830+0000 mon.a (mon.0) 1716 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:00.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[27669]: cluster 2024-02-18T01:25:58.936134+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-18T01:26:00.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[27669]: audit 2024-02-18T01:25:58.939490+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:00.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[27669]: audit 2024-02-18T01:25:58.941738+0000 mon.a (mon.0) 1718 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:00.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[19734]: audit 2024-02-18T01:25:58.935830+0000 mon.a (mon.0) 1716 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:00.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[19734]: cluster 2024-02-18T01:25:58.936134+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-18T01:26:00.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[19734]: audit 2024-02-18T01:25:58.939490+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:00.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:25:59 smithi105 bash[19734]: audit 2024-02-18T01:25:58.941738+0000 mon.a (mon.0) 1718 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:59 smithi155 bash[20805]: audit 2024-02-18T01:25:58.935830+0000 mon.a (mon.0) 1716 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:59 smithi155 bash[20805]: cluster 2024-02-18T01:25:58.936134+0000 mon.a (mon.0) 1717 : cluster [DBG] osdmap e233: 8 total, 8 up, 8 in 2024-02-18T01:26:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:59 smithi155 bash[20805]: audit 2024-02-18T01:25:58.939490+0000 mon.b (mon.1) 216 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:00.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:25:59 smithi155 bash[20805]: audit 2024-02-18T01:25:58.941738+0000 mon.a (mon.0) 1718 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]: dispatch 2024-02-18T01:26:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: cluster 2024-02-18T01:25:59.700245+0000 mgr.y (mgr.24475) 366 : cluster [DBG] pgmap v427: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: cluster 2024-02-18T01:25:59.930841+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:25:59.941634+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]': finished 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: cluster 2024-02-18T01:25:59.941827+0000 mon.a (mon.0) 1721 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:25:59.948186+0000 mon.c (mon.2) 339 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:25:59.949392+0000 mon.c (mon.2) 340 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:25:59.953740+0000 mon.a (mon.0) 1722 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:25:59.953972+0000 mon.a (mon.0) 1723 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:26:00.095633+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[19734]: audit 2024-02-18T01:26:00.095969+0000 mon.a (mon.0) 1724 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: cluster 2024-02-18T01:25:59.700245+0000 mgr.y (mgr.24475) 366 : cluster [DBG] pgmap v427: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: cluster 2024-02-18T01:25:59.930841+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:25:59.941634+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]': finished 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: cluster 2024-02-18T01:25:59.941827+0000 mon.a (mon.0) 1721 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-18T01:26:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:25:59.948186+0000 mon.c (mon.2) 339 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:25:59.949392+0000 mon.c (mon.2) 340 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:25:59.953740+0000 mon.a (mon.0) 1722 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:25:59.953972+0000 mon.a (mon.0) 1723 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:26:00.095633+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:01.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:00 smithi105 bash[27669]: audit 2024-02-18T01:26:00.095969+0000 mon.a (mon.0) 1724 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: cluster 2024-02-18T01:25:59.700245+0000 mgr.y (mgr.24475) 366 : cluster [DBG] pgmap v427: 244 pgs: 16 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: cluster 2024-02-18T01:25:59.930841+0000 mon.a (mon.0) 1719 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:25:59.941634+0000 mon.a (mon.0) 1720 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-31", "mode": "writeback"}]': finished 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: cluster 2024-02-18T01:25:59.941827+0000 mon.a (mon.0) 1721 : cluster [DBG] osdmap e234: 8 total, 8 up, 8 in 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:25:59.948186+0000 mon.c (mon.2) 339 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:25:59.949392+0000 mon.c (mon.2) 340 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:25:59.953740+0000 mon.a (mon.0) 1722 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:25:59.953972+0000 mon.a (mon.0) 1723 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:26:00.095633+0000 mon.b (mon.1) 217 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:00 smithi155 bash[20805]: audit 2024-02-18T01:26:00.095969+0000 mon.a (mon.0) 1724 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.955315+0000 mon.a (mon.0) 1725 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.955415+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.955510+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: cluster 2024-02-18T01:26:00.955585+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.958594+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.958732+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.960466+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.960668+0000 mon.c (mon.2) 342 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.967132+0000 mon.a (mon.0) 1730 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:00.967485+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: cluster 2024-02-18T01:26:01.952855+0000 mon.a (mon.0) 1732 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:02.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[19734]: audit 2024-02-18T01:26:01.962154+0000 mon.a (mon.0) 1733 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.955315+0000 mon.a (mon.0) 1725 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.955415+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.955510+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: cluster 2024-02-18T01:26:00.955585+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.958594+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.958732+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.960466+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.960668+0000 mon.c (mon.2) 342 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.967132+0000 mon.a (mon.0) 1730 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:00.967485+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: cluster 2024-02-18T01:26:01.952855+0000 mon.a (mon.0) 1732 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:02.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:01 smithi105 bash[27669]: audit 2024-02-18T01:26:01.962154+0000 mon.a (mon.0) 1733 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.955315+0000 mon.a (mon.0) 1725 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.955415+0000 mon.a (mon.0) 1726 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.955510+0000 mon.a (mon.0) 1727 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: cluster 2024-02-18T01:26:00.955585+0000 mon.a (mon.0) 1728 : cluster [DBG] osdmap e235: 8 total, 8 up, 8 in 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.958594+0000 mon.b (mon.1) 218 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.958732+0000 mon.a (mon.0) 1729 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]: dispatch 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.960466+0000 mon.c (mon.2) 341 : audit [INF] from='client.? v1:172.21.15.105:0/3006846419' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.960668+0000 mon.c (mon.2) 342 : audit [INF] from='client.? v1:172.21.15.105:0/3368745651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.967132+0000 mon.a (mon.0) 1730 : audit [INF] from='client.48110 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]: dispatch 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:00.967485+0000 mon.a (mon.0) 1731 : audit [INF] from='client.48113 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]: dispatch 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: cluster 2024-02-18T01:26:01.952855+0000 mon.a (mon.0) 1732 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:01 smithi155 bash[20805]: audit 2024-02-18T01:26:01.962154+0000 mon.a (mon.0) 1733 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-31"}]': finished 2024-02-18T01:26:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: cluster 2024-02-18T01:26:01.701069+0000 mgr.y (mgr.24475) 367 : cluster [DBG] pgmap v430: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.962362+0000 mon.a (mon.0) 1734 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.962539+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: cluster 2024-02-18T01:26:01.962692+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.992934+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.993704+0000 mon.c (mon.2) 343 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.994348+0000 mon.a (mon.0) 1738 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.994582+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.995020+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.995109+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.995414+0000 mon.a (mon.0) 1741 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.995919+0000 mon.c (mon.2) 345 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[27669]: audit 2024-02-18T01:26:01.996196+0000 mon.a (mon.0) 1742 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: cluster 2024-02-18T01:26:01.701069+0000 mgr.y (mgr.24475) 367 : cluster [DBG] pgmap v430: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.962362+0000 mon.a (mon.0) 1734 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.962539+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: cluster 2024-02-18T01:26:01.962692+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.992934+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.993704+0000 mon.c (mon.2) 343 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.994348+0000 mon.a (mon.0) 1738 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.994582+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.995020+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.995109+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.995414+0000 mon.a (mon.0) 1741 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.995919+0000 mon.c (mon.2) 345 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:02 smithi105 bash[19734]: audit 2024-02-18T01:26:01.996196+0000 mon.a (mon.0) 1742 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: cluster 2024-02-18T01:26:01.701069+0000 mgr.y (mgr.24475) 367 : cluster [DBG] pgmap v430: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.962362+0000 mon.a (mon.0) 1734 : audit [INF] from='client.48110 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi105-61649-38"}]': finished 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.962539+0000 mon.a (mon.0) 1735 : audit [INF] from='client.48113 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi105-61657-47"}]': finished 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: cluster 2024-02-18T01:26:01.962692+0000 mon.a (mon.0) 1736 : cluster [DBG] osdmap e236: 8 total, 8 up, 8 in 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.992934+0000 mon.a (mon.0) 1737 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.993704+0000 mon.c (mon.2) 343 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.994348+0000 mon.a (mon.0) 1738 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.994582+0000 mon.a (mon.0) 1739 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.995020+0000 mon.a (mon.0) 1740 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.995109+0000 mon.c (mon.2) 344 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.995414+0000 mon.a (mon.0) 1741 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.995919+0000 mon.c (mon.2) 345 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:03.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:02 smithi155 bash[20805]: audit 2024-02-18T01:26:01.996196+0000 mon.a (mon.0) 1742 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: audit 2024-02-18T01:26:02.991477+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: audit 2024-02-18T01:26:02.991638+0000 mon.a (mon.0) 1744 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: cluster 2024-02-18T01:26:02.991773+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: audit 2024-02-18T01:26:02.992572+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: audit 2024-02-18T01:26:02.993623+0000 mon.c (mon.2) 346 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[27669]: audit 2024-02-18T01:26:02.994417+0000 mon.a (mon.0) 1747 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: audit 2024-02-18T01:26:02.991477+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: audit 2024-02-18T01:26:02.991638+0000 mon.a (mon.0) 1744 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: cluster 2024-02-18T01:26:02.991773+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: audit 2024-02-18T01:26:02.992572+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: audit 2024-02-18T01:26:02.993623+0000 mon.c (mon.2) 346 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:04 smithi105 bash[19734]: audit 2024-02-18T01:26:02.994417+0000 mon.a (mon.0) 1747 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: audit 2024-02-18T01:26:02.991477+0000 mon.a (mon.0) 1743 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi105-61657-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: audit 2024-02-18T01:26:02.991638+0000 mon.a (mon.0) 1744 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi105-61649-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: cluster 2024-02-18T01:26:02.991773+0000 mon.a (mon.0) 1745 : cluster [DBG] osdmap e237: 8 total, 8 up, 8 in 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: audit 2024-02-18T01:26:02.992572+0000 mon.a (mon.0) 1746 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: audit 2024-02-18T01:26:02.993623+0000 mon.c (mon.2) 346 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:04.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:04 smithi155 bash[20805]: audit 2024-02-18T01:26:02.994417+0000 mon.a (mon.0) 1747 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[19734]: cluster 2024-02-18T01:26:03.701875+0000 mgr.y (mgr.24475) 368 : cluster [DBG] pgmap v433: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[19734]: cluster 2024-02-18T01:26:04.839732+0000 mon.a (mon.0) 1748 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-18T01:26:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[19734]: audit 2024-02-18T01:26:04.841965+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[19734]: audit 2024-02-18T01:26:04.844442+0000 mon.a (mon.0) 1749 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[27669]: cluster 2024-02-18T01:26:03.701875+0000 mgr.y (mgr.24475) 368 : cluster [DBG] pgmap v433: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[27669]: cluster 2024-02-18T01:26:04.839732+0000 mon.a (mon.0) 1748 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-18T01:26:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[27669]: audit 2024-02-18T01:26:04.841965+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:05.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:05 smithi105 bash[27669]: audit 2024-02-18T01:26:04.844442+0000 mon.a (mon.0) 1749 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:05.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:05 smithi155 bash[20805]: cluster 2024-02-18T01:26:03.701875+0000 mgr.y (mgr.24475) 368 : cluster [DBG] pgmap v433: 196 pgs: 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:05.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:05 smithi155 bash[20805]: cluster 2024-02-18T01:26:04.839732+0000 mon.a (mon.0) 1748 : cluster [DBG] osdmap e238: 8 total, 8 up, 8 in 2024-02-18T01:26:05.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:05 smithi155 bash[20805]: audit 2024-02-18T01:26:04.841965+0000 mon.b (mon.1) 219 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:05.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:05 smithi155 bash[20805]: audit 2024-02-18T01:26:04.844442+0000 mon.a (mon.0) 1749 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: audit 2024-02-18T01:26:05.019847+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: audit 2024-02-18T01:26:05.020167+0000 mon.a (mon.0) 1751 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: audit 2024-02-18T01:26:05.020597+0000 mon.a (mon.0) 1752 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: cluster 2024-02-18T01:26:05.020762+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-18T01:26:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: audit 2024-02-18T01:26:05.026351+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: audit 2024-02-18T01:26:05.030126+0000 mon.a (mon.0) 1754 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[27669]: cluster 2024-02-18T01:26:05.702714+0000 mgr.y (mgr.24475) 369 : cluster [DBG] pgmap v436: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: audit 2024-02-18T01:26:05.019847+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: audit 2024-02-18T01:26:05.020167+0000 mon.a (mon.0) 1751 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: audit 2024-02-18T01:26:05.020597+0000 mon.a (mon.0) 1752 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: cluster 2024-02-18T01:26:05.020762+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: audit 2024-02-18T01:26:05.026351+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: audit 2024-02-18T01:26:05.030126+0000 mon.a (mon.0) 1754 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[19734]: cluster 2024-02-18T01:26:05.702714+0000 mgr.y (mgr.24475) 369 : cluster [DBG] pgmap v436: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: audit 2024-02-18T01:26:05.019847+0000 mon.a (mon.0) 1750 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi105-61657-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: audit 2024-02-18T01:26:05.020167+0000 mon.a (mon.0) 1751 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi105-61649-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: audit 2024-02-18T01:26:05.020597+0000 mon.a (mon.0) 1752 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: cluster 2024-02-18T01:26:05.020762+0000 mon.a (mon.0) 1753 : cluster [DBG] osdmap e239: 8 total, 8 up, 8 in 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: audit 2024-02-18T01:26:05.026351+0000 mon.b (mon.1) 220 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: audit 2024-02-18T01:26:05.030126+0000 mon.a (mon.0) 1754 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:06 smithi155 bash[20805]: cluster 2024-02-18T01:26:05.702714+0000 mgr.y (mgr.24475) 369 : cluster [DBG] pgmap v436: 244 pgs: 48 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:06] "GET /metrics HTTP/1.1" 200 191674 "" "Prometheus/2.33.4" 2024-02-18T01:26:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:06.026015+0000 mon.a (mon.0) 1755 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: cluster 2024-02-18T01:26:06.026189+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:06.027337+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:06.028169+0000 mon.a (mon.0) 1757 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.025285+0000 mon.a (mon.0) 1758 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: cluster 2024-02-18T01:26:07.025594+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.027900+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.027949+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.028338+0000 mon.a (mon.0) 1761 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.029191+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[19734]: audit 2024-02-18T01:26:07.029328+0000 mon.a (mon.0) 1762 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:06.026015+0000 mon.a (mon.0) 1755 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: cluster 2024-02-18T01:26:06.026189+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:06.027337+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:06.028169+0000 mon.a (mon.0) 1757 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.025285+0000 mon.a (mon.0) 1758 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: cluster 2024-02-18T01:26:07.025594+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.027900+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.027949+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.028338+0000 mon.a (mon.0) 1761 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.029191+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:07 smithi105 bash[27669]: audit 2024-02-18T01:26:07.029328+0000 mon.a (mon.0) 1762 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:06.026015+0000 mon.a (mon.0) 1755 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: cluster 2024-02-18T01:26:06.026189+0000 mon.a (mon.0) 1756 : cluster [DBG] osdmap e240: 8 total, 8 up, 8 in 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:06.027337+0000 mon.b (mon.1) 221 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:06.028169+0000 mon.a (mon.0) 1757 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.025285+0000 mon.a (mon.0) 1758 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: cluster 2024-02-18T01:26:07.025594+0000 mon.a (mon.0) 1759 : cluster [DBG] osdmap e241: 8 total, 8 up, 8 in 2024-02-18T01:26:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.027900+0000 mon.c (mon.2) 347 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.027949+0000 mon.a (mon.0) 1760 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:07.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.028338+0000 mon.a (mon.0) 1761 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:07.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.029191+0000 mon.b (mon.1) 222 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[20805]: audit 2024-02-18T01:26:07.029328+0000 mon.a (mon.0) 1762 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]: dispatch 2024-02-18T01:26:07.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: cluster 2024-02-18T01:26:07.703802+0000 mgr.y (mgr.24475) 370 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:07.725803+0000 mgr.y (mgr.24475) 371 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: cluster 2024-02-18T01:26:08.019199+0000 mon.a (mon.0) 1763 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.028470+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.028661+0000 mon.a (mon.0) 1765 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.028890+0000 mon.a (mon.0) 1766 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]': finished 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: cluster 2024-02-18T01:26:08.029109+0000 mon.a (mon.0) 1767 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.029869+0000 mon.c (mon.2) 348 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.030969+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[19734]: audit 2024-02-18T01:26:08.031792+0000 mon.a (mon.0) 1769 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: cluster 2024-02-18T01:26:07.703802+0000 mgr.y (mgr.24475) 370 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:07.725803+0000 mgr.y (mgr.24475) 371 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: cluster 2024-02-18T01:26:08.019199+0000 mon.a (mon.0) 1763 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.028470+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.028661+0000 mon.a (mon.0) 1765 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.028890+0000 mon.a (mon.0) 1766 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]': finished 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: cluster 2024-02-18T01:26:08.029109+0000 mon.a (mon.0) 1767 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.029869+0000 mon.c (mon.2) 348 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.030969+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:08.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:08 smithi105 bash[27669]: audit 2024-02-18T01:26:08.031792+0000 mon.a (mon.0) 1769 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: cluster 2024-02-18T01:26:07.703802+0000 mgr.y (mgr.24475) 370 : cluster [DBG] pgmap v439: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:07.725803+0000 mgr.y (mgr.24475) 371 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: cluster 2024-02-18T01:26:08.019199+0000 mon.a (mon.0) 1763 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.028470+0000 mon.a (mon.0) 1764 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.028661+0000 mon.a (mon.0) 1765 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.028890+0000 mon.a (mon.0) 1766 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-33", "mode": "writeback"}]': finished 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: cluster 2024-02-18T01:26:08.029109+0000 mon.a (mon.0) 1767 : cluster [DBG] osdmap e242: 8 total, 8 up, 8 in 2024-02-18T01:26:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.029869+0000 mon.c (mon.2) 348 : audit [INF] from='client.? v1:172.21.15.105:0/1419260486' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:08.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.030969+0000 mon.a (mon.0) 1768 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]: dispatch 2024-02-18T01:26:08.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:08 smithi155 bash[20805]: audit 2024-02-18T01:26:08.031792+0000 mon.a (mon.0) 1769 : audit [INF] from='client.48125 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.031636+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.031739+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: cluster 2024-02-18T01:26:09.031844+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.055120+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.055136+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.057000+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.057161+0000 mon.a (mon.0) 1774 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.058331+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.058408+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.058628+0000 mon.c (mon.2) 349 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.059208+0000 mon.a (mon.0) 1776 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.060229+0000 mon.c (mon.2) 350 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.060545+0000 mon.a (mon.0) 1777 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.061079+0000 mon.c (mon.2) 351 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: audit 2024-02-18T01:26:09.061402+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[19734]: cluster 2024-02-18T01:26:09.704610+0000 mgr.y (mgr.24475) 372 : cluster [DBG] pgmap v442: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.031636+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.031739+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: cluster 2024-02-18T01:26:09.031844+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-18T01:26:10.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.055120+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.055136+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.057000+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.057161+0000 mon.a (mon.0) 1774 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.058331+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.058408+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.058628+0000 mon.c (mon.2) 349 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.059208+0000 mon.a (mon.0) 1776 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.060229+0000 mon.c (mon.2) 350 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.060545+0000 mon.a (mon.0) 1777 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.061079+0000 mon.c (mon.2) 351 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: audit 2024-02-18T01:26:09.061402+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:10 smithi105 bash[27669]: cluster 2024-02-18T01:26:09.704610+0000 mgr.y (mgr.24475) 372 : cluster [DBG] pgmap v442: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.031636+0000 mon.a (mon.0) 1770 : audit [INF] from='client.? v1:172.21.15.105:0/1751150055' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi105-61657-48"}]': finished 2024-02-18T01:26:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.031739+0000 mon.a (mon.0) 1771 : audit [INF] from='client.48125 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi105-61649-39"}]': finished 2024-02-18T01:26:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: cluster 2024-02-18T01:26:09.031844+0000 mon.a (mon.0) 1772 : cluster [DBG] osdmap e243: 8 total, 8 up, 8 in 2024-02-18T01:26:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.055120+0000 mon.b (mon.1) 223 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.055136+0000 mon.a (mon.0) 1773 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.057000+0000 mon.b (mon.1) 224 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.057161+0000 mon.a (mon.0) 1774 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.058331+0000 mon.b (mon.1) 225 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.058408+0000 mon.a (mon.0) 1775 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.058628+0000 mon.c (mon.2) 349 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.059208+0000 mon.a (mon.0) 1776 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.060229+0000 mon.c (mon.2) 350 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.060545+0000 mon.a (mon.0) 1777 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.061079+0000 mon.c (mon.2) 351 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: audit 2024-02-18T01:26:09.061402+0000 mon.a (mon.0) 1778 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:10.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:10 smithi155 bash[20805]: cluster 2024-02-18T01:26:09.704610+0000 mgr.y (mgr.24475) 372 : cluster [DBG] pgmap v442: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.046982+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.047126+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: cluster 2024-02-18T01:26:10.047241+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.048482+0000 mon.c (mon.2) 352 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.049008+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.049782+0000 mon.a (mon.0) 1782 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.050420+0000 mon.a (mon.0) 1783 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.110515+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[27669]: audit 2024-02-18T01:26:10.110753+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.046982+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.047126+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: cluster 2024-02-18T01:26:10.047241+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-18T01:26:12.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.048482+0000 mon.c (mon.2) 352 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.049008+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.049782+0000 mon.a (mon.0) 1782 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.050420+0000 mon.a (mon.0) 1783 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.110515+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:12.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:11 smithi105 bash[19734]: audit 2024-02-18T01:26:10.110753+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.046982+0000 mon.a (mon.0) 1779 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi105-61649-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.047126+0000 mon.a (mon.0) 1780 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi105-61657-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: cluster 2024-02-18T01:26:10.047241+0000 mon.a (mon.0) 1781 : cluster [DBG] osdmap e244: 8 total, 8 up, 8 in 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.048482+0000 mon.c (mon.2) 352 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.049008+0000 mon.b (mon.1) 226 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.049782+0000 mon.a (mon.0) 1782 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.050420+0000 mon.a (mon.0) 1783 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.110515+0000 mon.b (mon.1) 227 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:12.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:11 smithi155 bash[20805]: audit 2024-02-18T01:26:10.110753+0000 mon.a (mon.0) 1784 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: cluster 2024-02-18T01:26:11.705456+0000 mgr.y (mgr.24475) 373 : cluster [DBG] pgmap v445: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:11.886022+0000 mon.a (mon.0) 1785 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: cluster 2024-02-18T01:26:11.886224+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:11.898227+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:11.898308+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:11.987438+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.064370+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.064521+0000 mon.a (mon.0) 1790 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.064679+0000 mon.a (mon.0) 1791 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: cluster 2024-02-18T01:26:12.064788+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.320198+0000 mon.a (mon.0) 1793 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.321871+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:26:13.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[27669]: audit 2024-02-18T01:26:12.331677+0000 mon.a (mon.0) 1795 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: cluster 2024-02-18T01:26:11.705456+0000 mgr.y (mgr.24475) 373 : cluster [DBG] pgmap v445: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:11.886022+0000 mon.a (mon.0) 1785 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: cluster 2024-02-18T01:26:11.886224+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:11.898227+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:11.898308+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:11.987438+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.064370+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.064521+0000 mon.a (mon.0) 1790 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.064679+0000 mon.a (mon.0) 1791 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: cluster 2024-02-18T01:26:12.064788+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.320198+0000 mon.a (mon.0) 1793 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:26:13.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.321871+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:26:13.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:12 smithi105 bash[19734]: audit 2024-02-18T01:26:12.331677+0000 mon.a (mon.0) 1795 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: cluster 2024-02-18T01:26:11.705456+0000 mgr.y (mgr.24475) 373 : cluster [DBG] pgmap v445: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:11.886022+0000 mon.a (mon.0) 1785 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: cluster 2024-02-18T01:26:11.886224+0000 mon.a (mon.0) 1786 : cluster [DBG] osdmap e245: 8 total, 8 up, 8 in 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:11.898227+0000 mon.b (mon.1) 228 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:11.898308+0000 mon.a (mon.0) 1787 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:11.987438+0000 mon.a (mon.0) 1788 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:26:13.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.064370+0000 mon.a (mon.0) 1789 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi105-61657-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.064521+0000 mon.a (mon.0) 1790 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi105-61649-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.064679+0000 mon.a (mon.0) 1791 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: cluster 2024-02-18T01:26:12.064788+0000 mon.a (mon.0) 1792 : cluster [DBG] osdmap e246: 8 total, 8 up, 8 in 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.320198+0000 mon.a (mon.0) 1793 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.321871+0000 mon.a (mon.0) 1794 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:26:13.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:12 smithi155 bash[20805]: audit 2024-02-18T01:26:12.331677+0000 mon.a (mon.0) 1795 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[19734]: cluster 2024-02-18T01:26:13.058504+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-18T01:26:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[19734]: audit 2024-02-18T01:26:13.095067+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[19734]: audit 2024-02-18T01:26:13.095073+0000 mon.a (mon.0) 1797 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[19734]: cluster 2024-02-18T01:26:13.706393+0000 mgr.y (mgr.24475) 374 : cluster [DBG] pgmap v448: 244 pgs: 16 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[27669]: cluster 2024-02-18T01:26:13.058504+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-18T01:26:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[27669]: audit 2024-02-18T01:26:13.095067+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[27669]: audit 2024-02-18T01:26:13.095073+0000 mon.a (mon.0) 1797 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:14 smithi105 bash[27669]: cluster 2024-02-18T01:26:13.706393+0000 mgr.y (mgr.24475) 374 : cluster [DBG] pgmap v448: 244 pgs: 16 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:14 smithi155 bash[20805]: cluster 2024-02-18T01:26:13.058504+0000 mon.a (mon.0) 1796 : cluster [DBG] osdmap e247: 8 total, 8 up, 8 in 2024-02-18T01:26:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:14 smithi155 bash[20805]: audit 2024-02-18T01:26:13.095067+0000 mon.b (mon.1) 229 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:14 smithi155 bash[20805]: audit 2024-02-18T01:26:13.095073+0000 mon.a (mon.0) 1797 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:14 smithi155 bash[20805]: cluster 2024-02-18T01:26:13.706393+0000 mgr.y (mgr.24475) 374 : cluster [DBG] pgmap v448: 244 pgs: 16 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.072893+0000 mon.a (mon.0) 1798 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: cluster 2024-02-18T01:26:14.072995+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-18T01:26:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.076307+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.076535+0000 mon.a (mon.0) 1800 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.076538+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.076606+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.077026+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[27669]: audit 2024-02-18T01:26:14.077593+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.072893+0000 mon.a (mon.0) 1798 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: cluster 2024-02-18T01:26:14.072995+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.076307+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.076535+0000 mon.a (mon.0) 1800 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.076538+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.076606+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:15.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.077026+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:15 smithi105 bash[19734]: audit 2024-02-18T01:26:14.077593+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.072893+0000 mon.a (mon.0) 1798 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: cluster 2024-02-18T01:26:14.072995+0000 mon.a (mon.0) 1799 : cluster [DBG] osdmap e248: 8 total, 8 up, 8 in 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.076307+0000 mon.b (mon.1) 230 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.076535+0000 mon.a (mon.0) 1800 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.076538+0000 mon.b (mon.1) 231 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.076606+0000 mon.c (mon.2) 353 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.077026+0000 mon.a (mon.0) 1801 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:15.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:15 smithi155 bash[20805]: audit 2024-02-18T01:26:14.077593+0000 mon.a (mon.0) 1802 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: cluster 2024-02-18T01:26:15.064565+0000 mon.a (mon.0) 1803 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.072084+0000 mon.a (mon.0) 1804 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.072272+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.072433+0000 mon.a (mon.0) 1806 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: cluster 2024-02-18T01:26:15.072566+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.073620+0000 mon.c (mon.2) 354 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.075078+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.076230+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: audit 2024-02-18T01:26:15.076626+0000 mon.a (mon.0) 1809 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[27669]: cluster 2024-02-18T01:26:15.707253+0000 mgr.y (mgr.24475) 375 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: cluster 2024-02-18T01:26:15.064565+0000 mon.a (mon.0) 1803 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.072084+0000 mon.a (mon.0) 1804 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.072272+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.072433+0000 mon.a (mon.0) 1806 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: cluster 2024-02-18T01:26:15.072566+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.073620+0000 mon.c (mon.2) 354 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.075078+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.076230+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: audit 2024-02-18T01:26:15.076626+0000 mon.a (mon.0) 1809 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[19734]: cluster 2024-02-18T01:26:15.707253+0000 mgr.y (mgr.24475) 375 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:16.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:16] "GET /metrics HTTP/1.1" 200 191674 "" "Prometheus/2.33.4" 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: cluster 2024-02-18T01:26:15.064565+0000 mon.a (mon.0) 1803 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.072084+0000 mon.a (mon.0) 1804 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-33"}]': finished 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.072272+0000 mon.a (mon.0) 1805 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.072433+0000 mon.a (mon.0) 1806 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: cluster 2024-02-18T01:26:15.072566+0000 mon.a (mon.0) 1807 : cluster [DBG] osdmap e249: 8 total, 8 up, 8 in 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.073620+0000 mon.c (mon.2) 354 : audit [INF] from='client.? v1:172.21.15.105:0/3743648748' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.075078+0000 mon.b (mon.1) 232 : audit [INF] from='client.? v1:172.21.15.105:0/1359471651' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.076230+0000 mon.a (mon.0) 1808 : audit [INF] from='client.48137 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]: dispatch 2024-02-18T01:26:16.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: audit 2024-02-18T01:26:15.076626+0000 mon.a (mon.0) 1809 : audit [INF] from='client.48865 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]: dispatch 2024-02-18T01:26:16.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:16 smithi155 bash[20805]: cluster 2024-02-18T01:26:15.707253+0000 mgr.y (mgr.24475) 375 : cluster [DBG] pgmap v451: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.077996+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.078208+0000 mon.a (mon.0) 1811 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: cluster 2024-02-18T01:26:16.078438+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.521557+0000 mon.c (mon.2) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.521866+0000 mon.a (mon.0) 1813 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.522666+0000 mon.c (mon.2) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.522926+0000 mon.a (mon.0) 1814 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.523419+0000 mon.c (mon.2) 357 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:16.945563+0000 mon.a (mon.0) 1815 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.083070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:17.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: cluster 2024-02-18T01:26:17.083161+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.084496+0000 mon.c (mon.2) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.084944+0000 mon.c (mon.2) 359 : audit [INF] from='client.? v1:172.21.15.105:0/3169398869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.085715+0000 mon.a (mon.0) 1818 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.087198+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.087590+0000 mon.a (mon.0) 1819 : audit [INF] from='client.48149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[19734]: audit 2024-02-18T01:26:17.088032+0000 mon.a (mon.0) 1820 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.077996+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.078208+0000 mon.a (mon.0) 1811 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: cluster 2024-02-18T01:26:16.078438+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-18T01:26:17.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.521557+0000 mon.c (mon.2) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.521866+0000 mon.a (mon.0) 1813 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.522666+0000 mon.c (mon.2) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.522926+0000 mon.a (mon.0) 1814 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.523419+0000 mon.c (mon.2) 357 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:16.945563+0000 mon.a (mon.0) 1815 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.083070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: cluster 2024-02-18T01:26:17.083161+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.084496+0000 mon.c (mon.2) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.084944+0000 mon.c (mon.2) 359 : audit [INF] from='client.? v1:172.21.15.105:0/3169398869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.085715+0000 mon.a (mon.0) 1818 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.087198+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.087590+0000 mon.a (mon.0) 1819 : audit [INF] from='client.48149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:17 smithi105 bash[27669]: audit 2024-02-18T01:26:17.088032+0000 mon.a (mon.0) 1820 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.077996+0000 mon.a (mon.0) 1810 : audit [INF] from='client.48137 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi105-61657-49"}]': finished 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.078208+0000 mon.a (mon.0) 1811 : audit [INF] from='client.48865 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi105-61649-40"}]': finished 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: cluster 2024-02-18T01:26:16.078438+0000 mon.a (mon.0) 1812 : cluster [DBG] osdmap e250: 8 total, 8 up, 8 in 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.521557+0000 mon.c (mon.2) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.521866+0000 mon.a (mon.0) 1813 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.522666+0000 mon.c (mon.2) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.522926+0000 mon.a (mon.0) 1814 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.523419+0000 mon.c (mon.2) 357 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:16.945563+0000 mon.a (mon.0) 1815 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.083070+0000 mon.a (mon.0) 1816 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi105-61649-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: cluster 2024-02-18T01:26:17.083161+0000 mon.a (mon.0) 1817 : cluster [DBG] osdmap e251: 8 total, 8 up, 8 in 2024-02-18T01:26:17.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.084496+0000 mon.c (mon.2) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.084944+0000 mon.c (mon.2) 359 : audit [INF] from='client.? v1:172.21.15.105:0/3169398869' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.085715+0000 mon.a (mon.0) 1818 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:17.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.087198+0000 mon.b (mon.1) 233 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.087590+0000 mon.a (mon.0) 1819 : audit [INF] from='client.48149 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:17.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:17 smithi155 bash[20805]: audit 2024-02-18T01:26:17.088032+0000 mon.a (mon.0) 1820 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:18.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: cluster 2024-02-18T01:26:17.709695+0000 mgr.y (mgr.24475) 376 : cluster [DBG] pgmap v454: 260 pgs: 64 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:18.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: audit 2024-02-18T01:26:17.737059+0000 mgr.y (mgr.24475) 377 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: audit 2024-02-18T01:26:18.505112+0000 mon.a (mon.0) 1821 : audit [INF] from='client.48149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: audit 2024-02-18T01:26:18.505235+0000 mon.a (mon.0) 1822 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: cluster 2024-02-18T01:26:18.505423+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: audit 2024-02-18T01:26:18.507293+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[27669]: audit 2024-02-18T01:26:18.507508+0000 mon.a (mon.0) 1824 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: cluster 2024-02-18T01:26:17.709695+0000 mgr.y (mgr.24475) 376 : cluster [DBG] pgmap v454: 260 pgs: 64 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: audit 2024-02-18T01:26:17.737059+0000 mgr.y (mgr.24475) 377 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: audit 2024-02-18T01:26:18.505112+0000 mon.a (mon.0) 1821 : audit [INF] from='client.48149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: audit 2024-02-18T01:26:18.505235+0000 mon.a (mon.0) 1822 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: cluster 2024-02-18T01:26:18.505423+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-18T01:26:18.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: audit 2024-02-18T01:26:18.507293+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:18.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:18 smithi105 bash[19734]: audit 2024-02-18T01:26:18.507508+0000 mon.a (mon.0) 1824 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: cluster 2024-02-18T01:26:17.709695+0000 mgr.y (mgr.24475) 376 : cluster [DBG] pgmap v454: 260 pgs: 64 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: audit 2024-02-18T01:26:17.737059+0000 mgr.y (mgr.24475) 377 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: audit 2024-02-18T01:26:18.505112+0000 mon.a (mon.0) 1821 : audit [INF] from='client.48149 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi105-61657-50","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: audit 2024-02-18T01:26:18.505235+0000 mon.a (mon.0) 1822 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: cluster 2024-02-18T01:26:18.505423+0000 mon.a (mon.0) 1823 : cluster [DBG] osdmap e252: 8 total, 8 up, 8 in 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: audit 2024-02-18T01:26:18.507293+0000 mon.b (mon.1) 234 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:18 smithi155 bash[20805]: audit 2024-02-18T01:26:18.507508+0000 mon.a (mon.0) 1824 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.877449+0000 mon.a (mon.0) 1825 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.877798+0000 mon.a (mon.0) 1826 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: cluster 2024-02-18T01:26:18.877996+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-18T01:26:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.880376+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.880672+0000 mon.a (mon.0) 1828 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.907595+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.908854+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[19734]: audit 2024-02-18T01:26:18.909188+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:20.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.877449+0000 mon.a (mon.0) 1825 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.877798+0000 mon.a (mon.0) 1826 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: cluster 2024-02-18T01:26:18.877996+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.880376+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.880672+0000 mon.a (mon.0) 1828 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.907595+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.908854+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:19 smithi105 bash[27669]: audit 2024-02-18T01:26:18.909188+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.877449+0000 mon.a (mon.0) 1825 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi105-61649-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.877798+0000 mon.a (mon.0) 1826 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: cluster 2024-02-18T01:26:18.877996+0000 mon.a (mon.0) 1827 : cluster [DBG] osdmap e253: 8 total, 8 up, 8 in 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.880376+0000 mon.b (mon.1) 235 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.880672+0000 mon.a (mon.0) 1828 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.907595+0000 mon.a (mon.0) 1829 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.908854+0000 mon.a (mon.0) 1830 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:20.247 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:19 smithi155 bash[20805]: audit 2024-02-18T01:26:18.909188+0000 mon.a (mon.0) 1831 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: cluster 2024-02-18T01:26:19.710590+0000 mgr.y (mgr.24475) 378 : cluster [DBG] pgmap v457: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:19.882655+0000 mon.a (mon.0) 1832 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:19.883124+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: cluster 2024-02-18T01:26:19.883203+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:19.884881+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:19.885762+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:19.894779+0000 mon.a (mon.0) 1836 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: cluster 2024-02-18T01:26:20.877275+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[19734]: audit 2024-02-18T01:26:21.305987+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]': finished 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: cluster 2024-02-18T01:26:19.710590+0000 mgr.y (mgr.24475) 378 : cluster [DBG] pgmap v457: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:19.882655+0000 mon.a (mon.0) 1832 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:19.883124+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: cluster 2024-02-18T01:26:19.883203+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-18T01:26:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:19.884881+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:19.885762+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:19.894779+0000 mon.a (mon.0) 1836 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: cluster 2024-02-18T01:26:20.877275+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:21.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:21 smithi105 bash[27669]: audit 2024-02-18T01:26:21.305987+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]': finished 2024-02-18T01:26:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: cluster 2024-02-18T01:26:19.710590+0000 mgr.y (mgr.24475) 378 : cluster [DBG] pgmap v457: 236 pgs: 8 unknown, 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:26:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:19.882655+0000 mon.a (mon.0) 1832 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:19.883124+0000 mon.a (mon.0) 1833 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: cluster 2024-02-18T01:26:19.883203+0000 mon.a (mon.0) 1834 : cluster [DBG] osdmap e254: 8 total, 8 up, 8 in 2024-02-18T01:26:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:19.884881+0000 mon.a (mon.0) 1835 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:19.885762+0000 mon.b (mon.1) 236 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:19.894779+0000 mon.a (mon.0) 1836 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]: dispatch 2024-02-18T01:26:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: cluster 2024-02-18T01:26:20.877275+0000 mon.a (mon.0) 1837 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:21.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[20805]: audit 2024-02-18T01:26:21.305987+0000 mon.a (mon.0) 1838 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-35", "mode": "writeback"}]': finished 2024-02-18T01:26:22.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: cluster 2024-02-18T01:26:21.306105+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.307234+0000 mon.c (mon.2) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.308574+0000 mon.a (mon.0) 1840 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.344808+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.344956+0000 mon.a (mon.0) 1841 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.507173+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: cluster 2024-02-18T01:26:21.711402+0000 mgr.y (mgr.24475) 379 : cluster [DBG] pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.889025+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.889129+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.889225+0000 mon.a (mon.0) 1845 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: cluster 2024-02-18T01:26:21.889300+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-18T01:26:22.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.892324+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.894042+0000 mon.a (mon.0) 1847 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.895365+0000 mon.c (mon.2) 361 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[19734]: audit 2024-02-18T01:26:21.895847+0000 mon.a (mon.0) 1848 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: cluster 2024-02-18T01:26:21.306105+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.307234+0000 mon.c (mon.2) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.308574+0000 mon.a (mon.0) 1840 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.344808+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.344956+0000 mon.a (mon.0) 1841 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.507173+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: cluster 2024-02-18T01:26:21.711402+0000 mgr.y (mgr.24475) 379 : cluster [DBG] pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.889025+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.889129+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.889225+0000 mon.a (mon.0) 1845 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: cluster 2024-02-18T01:26:21.889300+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.892324+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.894042+0000 mon.a (mon.0) 1847 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.895365+0000 mon.c (mon.2) 361 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:22 smithi105 bash[27669]: audit 2024-02-18T01:26:21.895847+0000 mon.a (mon.0) 1848 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: cluster 2024-02-18T01:26:21.306105+0000 mon.a (mon.0) 1839 : cluster [DBG] osdmap e255: 8 total, 8 up, 8 in 2024-02-18T01:26:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.307234+0000 mon.c (mon.2) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.308574+0000 mon.a (mon.0) 1840 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.344808+0000 mon.b (mon.1) 237 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.344956+0000 mon.a (mon.0) 1841 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.507173+0000 mon.a (mon.0) 1842 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: cluster 2024-02-18T01:26:21.711402+0000 mgr.y (mgr.24475) 379 : cluster [DBG] pgmap v460: 228 pgs: 32 creating+peering, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.889025+0000 mon.a (mon.0) 1843 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi105-61657-51", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.889129+0000 mon.a (mon.0) 1844 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.889225+0000 mon.a (mon.0) 1845 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: cluster 2024-02-18T01:26:21.889300+0000 mon.a (mon.0) 1846 : cluster [DBG] osdmap e256: 8 total, 8 up, 8 in 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.892324+0000 mon.b (mon.1) 238 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.894042+0000 mon.a (mon.0) 1847 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]: dispatch 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.895365+0000 mon.c (mon.2) 361 : audit [INF] from='client.? v1:172.21.15.105:0/2834906968' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:22 smithi155 bash[20805]: audit 2024-02-18T01:26:21.895847+0000 mon.a (mon.0) 1848 : audit [INF] from='client.48146 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]: dispatch 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout:ete 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7021 ms) 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6959 ms) 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (6975 ms) 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7029 ms) 2024-02-18T01:26:22.904 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7024 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7041 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7027 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6945 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7024 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7051 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6955 ms) 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-02-18T01:26:22.905 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7054 ms) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7071 ms) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7044 ms) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (6819 ms) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112086 ms total) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [----------] Global test environment tear-down 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (202952 ms total) 2024-02-18T01:26:22.906 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio: [ PASSED ] 41 tests. 2024-02-18T01:26:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[27669]: cluster 2024-02-18T01:26:22.885438+0000 mon.a (mon.0) 1849 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[27669]: audit 2024-02-18T01:26:22.897325+0000 mon.a (mon.0) 1850 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[27669]: audit 2024-02-18T01:26:22.897486+0000 mon.a (mon.0) 1851 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:23.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[27669]: cluster 2024-02-18T01:26:22.897575+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-18T01:26:23.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[19734]: cluster 2024-02-18T01:26:22.885438+0000 mon.a (mon.0) 1849 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:23.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[19734]: audit 2024-02-18T01:26:22.897325+0000 mon.a (mon.0) 1850 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:23.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[19734]: audit 2024-02-18T01:26:22.897486+0000 mon.a (mon.0) 1851 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:23.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:23 smithi105 bash[19734]: cluster 2024-02-18T01:26:22.897575+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-18T01:26:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:23 smithi155 bash[20805]: cluster 2024-02-18T01:26:22.885438+0000 mon.a (mon.0) 1849 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:23 smithi155 bash[20805]: audit 2024-02-18T01:26:22.897325+0000 mon.a (mon.0) 1850 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-35"}]': finished 2024-02-18T01:26:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:23 smithi155 bash[20805]: audit 2024-02-18T01:26:22.897486+0000 mon.a (mon.0) 1851 : audit [INF] from='client.48146 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi105-61649-41"}]': finished 2024-02-18T01:26:23.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:23 smithi155 bash[20805]: cluster 2024-02-18T01:26:22.897575+0000 mon.a (mon.0) 1852 : cluster [DBG] osdmap e257: 8 total, 8 up, 8 in 2024-02-18T01:26:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[19734]: cluster 2024-02-18T01:26:23.712444+0000 mgr.y (mgr.24475) 380 : cluster [DBG] pgmap v463: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T01:26:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[19734]: cluster 2024-02-18T01:26:23.897366+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-18T01:26:25.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[19734]: audit 2024-02-18T01:26:23.899678+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:25.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[27669]: cluster 2024-02-18T01:26:23.712444+0000 mgr.y (mgr.24475) 380 : cluster [DBG] pgmap v463: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T01:26:25.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[27669]: cluster 2024-02-18T01:26:23.897366+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-18T01:26:25.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:24 smithi105 bash[27669]: audit 2024-02-18T01:26:23.899678+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:24 smithi155 bash[20805]: cluster 2024-02-18T01:26:23.712444+0000 mgr.y (mgr.24475) 380 : cluster [DBG] pgmap v463: 236 pgs: 8 creating+peering, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.2 KiB/s wr, 3 op/s 2024-02-18T01:26:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:24 smithi155 bash[20805]: cluster 2024-02-18T01:26:23.897366+0000 mon.a (mon.0) 1853 : cluster [DBG] osdmap e258: 8 total, 8 up, 8 in 2024-02-18T01:26:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:24 smithi155 bash[20805]: audit 2024-02-18T01:26:23.899678+0000 mon.a (mon.0) 1854 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[19734]: audit 2024-02-18T01:26:24.907843+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[19734]: cluster 2024-02-18T01:26:24.908798+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-18T01:26:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[19734]: audit 2024-02-18T01:26:24.914001+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[19734]: audit 2024-02-18T01:26:24.915139+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[19734]: audit 2024-02-18T01:26:24.916007+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[27669]: audit 2024-02-18T01:26:24.907843+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[27669]: cluster 2024-02-18T01:26:24.908798+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[27669]: audit 2024-02-18T01:26:24.914001+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[27669]: audit 2024-02-18T01:26:24.915139+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:25 smithi105 bash[27669]: audit 2024-02-18T01:26:24.916007+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:25 smithi155 bash[20805]: audit 2024-02-18T01:26:24.907843+0000 mon.a (mon.0) 1855 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:25 smithi155 bash[20805]: cluster 2024-02-18T01:26:24.908798+0000 mon.a (mon.0) 1856 : cluster [DBG] osdmap e259: 8 total, 8 up, 8 in 2024-02-18T01:26:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:25 smithi155 bash[20805]: audit 2024-02-18T01:26:24.914001+0000 mon.a (mon.0) 1857 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]: dispatch 2024-02-18T01:26:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:25 smithi155 bash[20805]: audit 2024-02-18T01:26:24.915139+0000 mon.b (mon.1) 239 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:25 smithi155 bash[20805]: audit 2024-02-18T01:26:24.916007+0000 mon.a (mon.0) 1858 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:26.932 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:26] "GET /metrics HTTP/1.1" 200 186970 "" "Prometheus/2.33.4" 2024-02-18T01:26:27.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: cluster 2024-02-18T01:26:25.713462+0000 mgr.y (mgr.24475) 381 : cluster [DBG] pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.914035+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.914175+0000 mon.a (mon.0) 1860 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: cluster 2024-02-18T01:26:25.914553+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.921587+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.923849+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.947212+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.948454+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:25.948821+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:26.915865+0000 mon.a (mon.0) 1866 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:26.915997+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: cluster 2024-02-18T01:26:26.916092+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:26.917751+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:26.920002+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[19734]: audit 2024-02-18T01:26:26.921503+0000 mon.a (mon.0) 1870 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: cluster 2024-02-18T01:26:25.713462+0000 mgr.y (mgr.24475) 381 : cluster [DBG] pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.914035+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.914175+0000 mon.a (mon.0) 1860 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: cluster 2024-02-18T01:26:25.914553+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.921587+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.923849+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.947212+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.948454+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:25.948821+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:27.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:26.915865+0000 mon.a (mon.0) 1866 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:27.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:26.915997+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:27.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: cluster 2024-02-18T01:26:26.916092+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-18T01:26:27.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:26.917751+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:26.920002+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:26 smithi105 bash[27669]: audit 2024-02-18T01:26:26.921503+0000 mon.a (mon.0) 1870 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: cluster 2024-02-18T01:26:25.713462+0000 mgr.y (mgr.24475) 381 : cluster [DBG] pgmap v466: 228 pgs: 32 unknown, 196 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.914035+0000 mon.a (mon.0) 1859 : audit [INF] from='client.? v1:172.21.15.105:0/1393795027' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi105-61657-51"}]': finished 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.914175+0000 mon.a (mon.0) 1860 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: cluster 2024-02-18T01:26:25.914553+0000 mon.a (mon.0) 1861 : cluster [DBG] osdmap e260: 8 total, 8 up, 8 in 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.921587+0000 mon.b (mon.1) 240 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.923849+0000 mon.a (mon.0) 1862 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.947212+0000 mon.a (mon.0) 1863 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.948454+0000 mon.a (mon.0) 1864 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:25.948821+0000 mon.a (mon.0) 1865 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:26.915865+0000 mon.a (mon.0) 1866 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:26.915997+0000 mon.a (mon.0) 1867 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: cluster 2024-02-18T01:26:26.916092+0000 mon.a (mon.0) 1868 : cluster [DBG] osdmap e261: 8 total, 8 up, 8 in 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:26.917751+0000 mon.b (mon.1) 241 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:26.920002+0000 mon.a (mon.0) 1869 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:27.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:26 smithi155 bash[20805]: audit 2024-02-18T01:26:26.921503+0000 mon.a (mon.0) 1870 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:27.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: cluster 2024-02-18T01:26:27.715642+0000 mgr.y (mgr.24475) 382 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: audit 2024-02-18T01:26:27.740536+0000 mgr.y (mgr.24475) 383 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: audit 2024-02-18T01:26:28.345931+0000 mon.a (mon.0) 1871 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: cluster 2024-02-18T01:26:28.346143+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: audit 2024-02-18T01:26:28.347699+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[19734]: audit 2024-02-18T01:26:28.348399+0000 mon.a (mon.0) 1873 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: cluster 2024-02-18T01:26:27.715642+0000 mgr.y (mgr.24475) 382 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: audit 2024-02-18T01:26:27.740536+0000 mgr.y (mgr.24475) 383 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: audit 2024-02-18T01:26:28.345931+0000 mon.a (mon.0) 1871 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: cluster 2024-02-18T01:26:28.346143+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: audit 2024-02-18T01:26:28.347699+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:29.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:28 smithi105 bash[27669]: audit 2024-02-18T01:26:28.348399+0000 mon.a (mon.0) 1873 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: cluster 2024-02-18T01:26:27.715642+0000 mgr.y (mgr.24475) 382 : cluster [DBG] pgmap v469: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: audit 2024-02-18T01:26:27.740536+0000 mgr.y (mgr.24475) 383 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: audit 2024-02-18T01:26:28.345931+0000 mon.a (mon.0) 1871 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: cluster 2024-02-18T01:26:28.346143+0000 mon.a (mon.0) 1872 : cluster [DBG] osdmap e262: 8 total, 8 up, 8 in 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: audit 2024-02-18T01:26:28.347699+0000 mon.b (mon.1) 242 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:29.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:28 smithi155 bash[20805]: audit 2024-02-18T01:26:28.348399+0000 mon.a (mon.0) 1873 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]: dispatch 2024-02-18T01:26:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: cluster 2024-02-18T01:26:28.911995+0000 mon.a (mon.0) 1874 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: audit 2024-02-18T01:26:28.921625+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: audit 2024-02-18T01:26:28.921957+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]': finished 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: cluster 2024-02-18T01:26:28.922311+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: audit 2024-02-18T01:26:29.025611+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[19734]: audit 2024-02-18T01:26:29.025844+0000 mon.a (mon.0) 1878 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: cluster 2024-02-18T01:26:28.911995+0000 mon.a (mon.0) 1874 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: audit 2024-02-18T01:26:28.921625+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: audit 2024-02-18T01:26:28.921957+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]': finished 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: cluster 2024-02-18T01:26:28.922311+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: audit 2024-02-18T01:26:29.025611+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:30.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:29 smithi105 bash[27669]: audit 2024-02-18T01:26:29.025844+0000 mon.a (mon.0) 1878 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: cluster 2024-02-18T01:26:28.911995+0000 mon.a (mon.0) 1874 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: audit 2024-02-18T01:26:28.921625+0000 mon.a (mon.0) 1875 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi105-61657-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: audit 2024-02-18T01:26:28.921957+0000 mon.a (mon.0) 1876 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-37", "mode": "writeback"}]': finished 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: cluster 2024-02-18T01:26:28.922311+0000 mon.a (mon.0) 1877 : cluster [DBG] osdmap e263: 8 total, 8 up, 8 in 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: audit 2024-02-18T01:26:29.025611+0000 mon.b (mon.1) 243 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:29 smithi155 bash[20805]: audit 2024-02-18T01:26:29.025844+0000 mon.a (mon.0) 1878 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: cluster 2024-02-18T01:26:29.716500+0000 mgr.y (mgr.24475) 384 : cluster [DBG] pgmap v472: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: audit 2024-02-18T01:26:29.925446+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: cluster 2024-02-18T01:26:29.925638+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: audit 2024-02-18T01:26:29.927189+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: audit 2024-02-18T01:26:29.930513+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: cluster 2024-02-18T01:26:30.917539+0000 mon.a (mon.0) 1882 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: audit 2024-02-18T01:26:30.926917+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: cluster 2024-02-18T01:26:30.927077+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[19734]: audit 2024-02-18T01:26:30.929715+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: cluster 2024-02-18T01:26:29.716500+0000 mgr.y (mgr.24475) 384 : cluster [DBG] pgmap v472: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: audit 2024-02-18T01:26:29.925446+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: cluster 2024-02-18T01:26:29.925638+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: audit 2024-02-18T01:26:29.927189+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: audit 2024-02-18T01:26:29.930513+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: cluster 2024-02-18T01:26:30.917539+0000 mon.a (mon.0) 1882 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:31.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: audit 2024-02-18T01:26:30.926917+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:31.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: cluster 2024-02-18T01:26:30.927077+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-18T01:26:31.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:30 smithi105 bash[27669]: audit 2024-02-18T01:26:30.929715+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: cluster 2024-02-18T01:26:29.716500+0000 mgr.y (mgr.24475) 384 : cluster [DBG] pgmap v472: 236 pgs: 8 unknown, 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: audit 2024-02-18T01:26:29.925446+0000 mon.a (mon.0) 1879 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: cluster 2024-02-18T01:26:29.925638+0000 mon.a (mon.0) 1880 : cluster [DBG] osdmap e264: 8 total, 8 up, 8 in 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: audit 2024-02-18T01:26:29.927189+0000 mon.b (mon.1) 244 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: audit 2024-02-18T01:26:29.930513+0000 mon.a (mon.0) 1881 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]: dispatch 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: cluster 2024-02-18T01:26:30.917539+0000 mon.a (mon.0) 1882 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: audit 2024-02-18T01:26:30.926917+0000 mon.a (mon.0) 1883 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-37"}]': finished 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: cluster 2024-02-18T01:26:30.927077+0000 mon.a (mon.0) 1884 : cluster [DBG] osdmap e265: 8 total, 8 up, 8 in 2024-02-18T01:26:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:30 smithi155 bash[20805]: audit 2024-02-18T01:26:30.929715+0000 mon.a (mon.0) 1885 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[19734]: cluster 2024-02-18T01:26:31.717282+0000 mgr.y (mgr.24475) 385 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[19734]: audit 2024-02-18T01:26:31.929331+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[19734]: cluster 2024-02-18T01:26:31.929676+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-18T01:26:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[19734]: audit 2024-02-18T01:26:31.930568+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:33.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[27669]: cluster 2024-02-18T01:26:31.717282+0000 mgr.y (mgr.24475) 385 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:33.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[27669]: audit 2024-02-18T01:26:31.929331+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:33.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[27669]: cluster 2024-02-18T01:26:31.929676+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-18T01:26:33.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:32 smithi105 bash[27669]: audit 2024-02-18T01:26:31.930568+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:32 smithi155 bash[20805]: cluster 2024-02-18T01:26:31.717282+0000 mgr.y (mgr.24475) 385 : cluster [DBG] pgmap v475: 228 pgs: 228 active+clean; 6.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:32 smithi155 bash[20805]: audit 2024-02-18T01:26:31.929331+0000 mon.a (mon.0) 1886 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:32 smithi155 bash[20805]: cluster 2024-02-18T01:26:31.929676+0000 mon.a (mon.0) 1887 : cluster [DBG] osdmap e266: 8 total, 8 up, 8 in 2024-02-18T01:26:33.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:32 smithi155 bash[20805]: audit 2024-02-18T01:26:31.930568+0000 mon.a (mon.0) 1888 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]: dispatch 2024-02-18T01:26:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.934699+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: cluster 2024-02-18T01:26:32.934969+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.941882+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.948473+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.964393+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.964770+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.965790+0000 mon.c (mon.2) 363 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.966126+0000 mon.a (mon.0) 1893 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.966657+0000 mon.c (mon.2) 364 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:32.966975+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:33.935598+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:33.935846+0000 mon.a (mon.0) 1896 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:34.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: cluster 2024-02-18T01:26:33.936039+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:33.939321+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[19734]: audit 2024-02-18T01:26:33.939961+0000 mon.a (mon.0) 1898 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.934699+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: cluster 2024-02-18T01:26:32.934969+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.941882+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.948473+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.964393+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.964770+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.965790+0000 mon.c (mon.2) 363 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.966126+0000 mon.a (mon.0) 1893 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.966657+0000 mon.c (mon.2) 364 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:32.966975+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:33.935598+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:33.935846+0000 mon.a (mon.0) 1896 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: cluster 2024-02-18T01:26:33.936039+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:33.939321+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:33 smithi105 bash[27669]: audit 2024-02-18T01:26:33.939961+0000 mon.a (mon.0) 1898 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.934699+0000 mon.a (mon.0) 1889 : audit [INF] from='client.? v1:172.21.15.105:0/3194047821' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi105-61657-52"}]': finished 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: cluster 2024-02-18T01:26:32.934969+0000 mon.a (mon.0) 1890 : cluster [DBG] osdmap e267: 8 total, 8 up, 8 in 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.941882+0000 mon.b (mon.1) 245 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.948473+0000 mon.a (mon.0) 1891 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.964393+0000 mon.c (mon.2) 362 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.964770+0000 mon.a (mon.0) 1892 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.965790+0000 mon.c (mon.2) 363 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.966126+0000 mon.a (mon.0) 1893 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.966657+0000 mon.c (mon.2) 364 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:32.966975+0000 mon.a (mon.0) 1894 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:33.935598+0000 mon.a (mon.0) 1895 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:33.935846+0000 mon.a (mon.0) 1896 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi105-61657-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: cluster 2024-02-18T01:26:33.936039+0000 mon.a (mon.0) 1897 : cluster [DBG] osdmap e268: 8 total, 8 up, 8 in 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:33.939321+0000 mon.c (mon.2) 365 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:34.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:33 smithi155 bash[20805]: audit 2024-02-18T01:26:33.939961+0000 mon.a (mon.0) 1898 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:35.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[19734]: cluster 2024-02-18T01:26:33.718131+0000 mgr.y (mgr.24475) 386 : cluster [DBG] pgmap v478: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:35.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[19734]: audit 2024-02-18T01:26:33.942528+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:35.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[19734]: audit 2024-02-18T01:26:33.943294+0000 mon.a (mon.0) 1899 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:35.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[27669]: cluster 2024-02-18T01:26:33.718131+0000 mgr.y (mgr.24475) 386 : cluster [DBG] pgmap v478: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:35.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[27669]: audit 2024-02-18T01:26:33.942528+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:35.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:35 smithi105 bash[27669]: audit 2024-02-18T01:26:33.943294+0000 mon.a (mon.0) 1899 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:35.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:35 smithi155 bash[20805]: cluster 2024-02-18T01:26:33.718131+0000 mgr.y (mgr.24475) 386 : cluster [DBG] pgmap v478: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:35.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:35 smithi155 bash[20805]: audit 2024-02-18T01:26:33.942528+0000 mon.b (mon.1) 246 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:35.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:35 smithi155 bash[20805]: audit 2024-02-18T01:26:33.943294+0000 mon.a (mon.0) 1899 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:36.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.362867+0000 mon.a (mon.0) 1900 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: cluster 2024-02-18T01:26:35.363120+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.364632+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.364991+0000 mon.a (mon.0) 1902 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: cluster 2024-02-18T01:26:35.718944+0000 mgr.y (mgr.24475) 387 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.942632+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:36.681 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.942959+0000 mon.a (mon.0) 1904 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:26:36.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: cluster 2024-02-18T01:26:35.943149+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-18T01:26:36.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.944957+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.682 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[27669]: audit 2024-02-18T01:26:35.946547+0000 mon.a (mon.0) 1906 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.682 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.362867+0000 mon.a (mon.0) 1900 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:36.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: cluster 2024-02-18T01:26:35.363120+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-18T01:26:36.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.364632+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.364991+0000 mon.a (mon.0) 1902 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: cluster 2024-02-18T01:26:35.718944+0000 mgr.y (mgr.24475) 387 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:36.683 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.942632+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:36.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.942959+0000 mon.a (mon.0) 1904 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:26:36.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: cluster 2024-02-18T01:26:35.943149+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-18T01:26:36.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.944957+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.684 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[19734]: audit 2024-02-18T01:26:35.946547+0000 mon.a (mon.0) 1906 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.362867+0000 mon.a (mon.0) 1900 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: cluster 2024-02-18T01:26:35.363120+0000 mon.a (mon.0) 1901 : cluster [DBG] osdmap e269: 8 total, 8 up, 8 in 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.364632+0000 mon.b (mon.1) 247 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.364991+0000 mon.a (mon.0) 1902 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: cluster 2024-02-18T01:26:35.718944+0000 mgr.y (mgr.24475) 387 : cluster [DBG] pgmap v481: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.942632+0000 mon.a (mon.0) 1903 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi105-61657-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:36.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.942959+0000 mon.a (mon.0) 1904 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:26:36.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: cluster 2024-02-18T01:26:35.943149+0000 mon.a (mon.0) 1905 : cluster [DBG] osdmap e270: 8 total, 8 up, 8 in 2024-02-18T01:26:36.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.944957+0000 mon.b (mon.1) 248 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:36 smithi155 bash[20805]: audit 2024-02-18T01:26:35.946547+0000 mon.a (mon.0) 1906 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:36] "GET /metrics HTTP/1.1" 200 189644 "" "Prometheus/2.33.4" 2024-02-18T01:26:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[27669]: audit 2024-02-18T01:26:36.509344+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[27669]: audit 2024-02-18T01:26:36.951005+0000 mon.a (mon.0) 1908 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[27669]: cluster 2024-02-18T01:26:36.951407+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-18T01:26:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[27669]: audit 2024-02-18T01:26:36.953431+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[27669]: audit 2024-02-18T01:26:36.953739+0000 mon.a (mon.0) 1910 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[19734]: audit 2024-02-18T01:26:36.509344+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[19734]: audit 2024-02-18T01:26:36.951005+0000 mon.a (mon.0) 1908 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[19734]: cluster 2024-02-18T01:26:36.951407+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-18T01:26:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[19734]: audit 2024-02-18T01:26:36.953431+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:37 smithi105 bash[19734]: audit 2024-02-18T01:26:36.953739+0000 mon.a (mon.0) 1910 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:37.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[20805]: audit 2024-02-18T01:26:36.509344+0000 mon.a (mon.0) 1907 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:26:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[20805]: audit 2024-02-18T01:26:36.951005+0000 mon.a (mon.0) 1908 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[20805]: cluster 2024-02-18T01:26:36.951407+0000 mon.a (mon.0) 1909 : cluster [DBG] osdmap e271: 8 total, 8 up, 8 in 2024-02-18T01:26:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[20805]: audit 2024-02-18T01:26:36.953431+0000 mon.b (mon.1) 249 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:37 smithi155 bash[20805]: audit 2024-02-18T01:26:36.953739+0000 mon.a (mon.0) 1910 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:26:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: cluster 2024-02-18T01:26:37.721249+0000 mgr.y (mgr.24475) 388 : cluster [DBG] pgmap v484: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: audit 2024-02-18T01:26:37.749844+0000 mgr.y (mgr.24475) 389 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: audit 2024-02-18T01:26:37.949012+0000 mon.a (mon.0) 1911 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:26:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: cluster 2024-02-18T01:26:37.949349+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-18T01:26:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: audit 2024-02-18T01:26:37.950859+0000 mon.c (mon.2) 366 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: audit 2024-02-18T01:26:37.951491+0000 mon.a (mon.0) 1913 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[19734]: cluster 2024-02-18T01:26:38.507840+0000 mon.a (mon.0) 1914 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: cluster 2024-02-18T01:26:37.721249+0000 mgr.y (mgr.24475) 388 : cluster [DBG] pgmap v484: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: audit 2024-02-18T01:26:37.749844+0000 mgr.y (mgr.24475) 389 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: audit 2024-02-18T01:26:37.949012+0000 mon.a (mon.0) 1911 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: cluster 2024-02-18T01:26:37.949349+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: audit 2024-02-18T01:26:37.950859+0000 mon.c (mon.2) 366 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: audit 2024-02-18T01:26:37.951491+0000 mon.a (mon.0) 1913 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:38 smithi105 bash[27669]: cluster 2024-02-18T01:26:38.507840+0000 mon.a (mon.0) 1914 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: cluster 2024-02-18T01:26:37.721249+0000 mgr.y (mgr.24475) 388 : cluster [DBG] pgmap v484: 236 pgs: 236 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: audit 2024-02-18T01:26:37.749844+0000 mgr.y (mgr.24475) 389 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: audit 2024-02-18T01:26:37.949012+0000 mon.a (mon.0) 1911 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-39","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: cluster 2024-02-18T01:26:37.949349+0000 mon.a (mon.0) 1912 : cluster [DBG] osdmap e272: 8 total, 8 up, 8 in 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: audit 2024-02-18T01:26:37.950859+0000 mon.c (mon.2) 366 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: audit 2024-02-18T01:26:37.951491+0000 mon.a (mon.0) 1913 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:38 smithi155 bash[20805]: cluster 2024-02-18T01:26:38.507840+0000 mon.a (mon.0) 1914 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.952314+0000 mon.a (mon.0) 1915 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: cluster 2024-02-18T01:26:38.952487+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.955148+0000 mon.c (mon.2) 367 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.956770+0000 mon.a (mon.0) 1917 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.983982+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.984097+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.985108+0000 mon.a (mon.0) 1919 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:38.985111+0000 mon.b (mon.1) 251 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:39.954544+0000 mon.a (mon.0) 1920 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: audit 2024-02-18T01:26:39.954677+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]': finished 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[19734]: cluster 2024-02-18T01:26:39.954780+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-18T01:26:40.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.952314+0000 mon.a (mon.0) 1915 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: cluster 2024-02-18T01:26:38.952487+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.955148+0000 mon.c (mon.2) 367 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.956770+0000 mon.a (mon.0) 1917 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.983982+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.984097+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.985108+0000 mon.a (mon.0) 1919 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:38.985111+0000 mon.b (mon.1) 251 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:39.954544+0000 mon.a (mon.0) 1920 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: audit 2024-02-18T01:26:39.954677+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]': finished 2024-02-18T01:26:40.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:39 smithi105 bash[27669]: cluster 2024-02-18T01:26:39.954780+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.952314+0000 mon.a (mon.0) 1915 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: cluster 2024-02-18T01:26:38.952487+0000 mon.a (mon.0) 1916 : cluster [DBG] osdmap e273: 8 total, 8 up, 8 in 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.955148+0000 mon.c (mon.2) 367 : audit [INF] from='client.? v1:172.21.15.105:0/3187006045' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.956770+0000 mon.a (mon.0) 1917 : audit [INF] from='client.48167 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.983982+0000 mon.b (mon.1) 250 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.984097+0000 mon.a (mon.0) 1918 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.985108+0000 mon.a (mon.0) 1919 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:38.985111+0000 mon.b (mon.1) 251 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]: dispatch 2024-02-18T01:26:40.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:39.954544+0000 mon.a (mon.0) 1920 : audit [INF] from='client.48167 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi105-61657-53"}]': finished 2024-02-18T01:26:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: audit 2024-02-18T01:26:39.954677+0000 mon.a (mon.0) 1921 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-39"}]': finished 2024-02-18T01:26:40.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:39 smithi155 bash[20805]: cluster 2024-02-18T01:26:39.954780+0000 mon.a (mon.0) 1922 : cluster [DBG] osdmap e274: 8 total, 8 up, 8 in 2024-02-18T01:26:40.982 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [==========] Running 72 tests from 4 test suites. 2024-02-18T01:26:40.982 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: seed 61833 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (317 ms) 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (15806 ms) 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (21 ms) 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (16144 ms total) 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-02-18T01:26:40.983 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7792 ms) 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8345 ms) 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (10324 ms) 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: my_snaps [4] 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: my_snaps [5,4] 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: my_snaps [6,5,4] 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: promoting some heads 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: promoting from clones for snap 7 2024-02-18T01:26:40.984 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: promoting from clones for snap 6 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: promoting from clones for snap 5 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: promoting from clones for snap 4 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: waiting for scrubs... 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: done waiting 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (45990 ms) 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10387 ms) 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7555 ms) 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8032 ms) 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8026 ms) 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-02-18T01:26:40.985 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10058 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8960 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10038 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7972 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8088 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13081 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7819 ms) 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-02-18T01:26:40.986 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8032 ms) 2024-02-18T01:26:40.987 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-02-18T01:26:40.987 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-18T01:26:40.987 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: ok, hit_set contains 188:602f83fe:::foo:head 2024-02-18T01:26:41.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[27669]: cluster 2024-02-18T01:26:39.722140+0000 mgr.y (mgr.24475) 390 : cluster [DBG] pgmap v487: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:41.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[27669]: audit 2024-02-18T01:26:39.977080+0000 mon.a (mon.0) 1923 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[27669]: audit 2024-02-18T01:26:39.979134+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[27669]: audit 2024-02-18T01:26:39.979638+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:41.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[19734]: cluster 2024-02-18T01:26:39.722140+0000 mgr.y (mgr.24475) 390 : cluster [DBG] pgmap v487: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:41.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[19734]: audit 2024-02-18T01:26:39.977080+0000 mon.a (mon.0) 1923 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[19734]: audit 2024-02-18T01:26:39.979134+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:40 smithi105 bash[19734]: audit 2024-02-18T01:26:39.979638+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:40 smithi155 bash[20805]: cluster 2024-02-18T01:26:39.722140+0000 mgr.y (mgr.24475) 390 : cluster [DBG] pgmap v487: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:40 smithi155 bash[20805]: audit 2024-02-18T01:26:39.977080+0000 mon.a (mon.0) 1923 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:40 smithi155 bash[20805]: audit 2024-02-18T01:26:39.979134+0000 mon.a (mon.0) 1924 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:41.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:40 smithi155 bash[20805]: audit 2024-02-18T01:26:39.979638+0000 mon.a (mon.0) 1925 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[19734]: audit 2024-02-18T01:26:40.979989+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[19734]: cluster 2024-02-18T01:26:40.981010+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-18T01:26:42.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[19734]: audit 2024-02-18T01:26:40.983376+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[27669]: audit 2024-02-18T01:26:40.979989+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[27669]: cluster 2024-02-18T01:26:40.981010+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-18T01:26:42.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:42 smithi105 bash[27669]: audit 2024-02-18T01:26:40.983376+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:42 smithi155 bash[20805]: audit 2024-02-18T01:26:40.979989+0000 mon.a (mon.0) 1926 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi105-61657-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:42 smithi155 bash[20805]: cluster 2024-02-18T01:26:40.981010+0000 mon.a (mon.0) 1927 : cluster [DBG] osdmap e275: 8 total, 8 up, 8 in 2024-02-18T01:26:42.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:42 smithi155 bash[20805]: audit 2024-02-18T01:26:40.983376+0000 mon.a (mon.0) 1928 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: cluster 2024-02-18T01:26:41.722948+0000 mgr.y (mgr.24475) 391 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: cluster 2024-02-18T01:26:42.406738+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-18T01:26:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.409026+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.415175+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.984784+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.985842+0000 mon.a (mon.0) 1932 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: cluster 2024-02-18T01:26:42.986059+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.990361+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.991638+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[27669]: audit 2024-02-18T01:26:42.993505+0000 mon.a (mon.0) 1934 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: cluster 2024-02-18T01:26:41.722948+0000 mgr.y (mgr.24475) 391 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: cluster 2024-02-18T01:26:42.406738+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.409026+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.415175+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.984784+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:43.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.985842+0000 mon.a (mon.0) 1932 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:43.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: cluster 2024-02-18T01:26:42.986059+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-18T01:26:43.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.990361+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:43.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.991638+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:43.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:43 smithi105 bash[19734]: audit 2024-02-18T01:26:42.993505+0000 mon.a (mon.0) 1934 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: cluster 2024-02-18T01:26:41.722948+0000 mgr.y (mgr.24475) 391 : cluster [DBG] pgmap v490: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: cluster 2024-02-18T01:26:42.406738+0000 mon.a (mon.0) 1929 : cluster [DBG] osdmap e276: 8 total, 8 up, 8 in 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.409026+0000 mon.b (mon.1) 252 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.415175+0000 mon.a (mon.0) 1930 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.984784+0000 mon.a (mon.0) 1931 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi105-61657-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.985842+0000 mon.a (mon.0) 1932 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: cluster 2024-02-18T01:26:42.986059+0000 mon.a (mon.0) 1933 : cluster [DBG] osdmap e277: 8 total, 8 up, 8 in 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.990361+0000 mon.b (mon.1) 253 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-6","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.991638+0000 mon.b (mon.1) 254 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:43.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:43 smithi155 bash[20805]: audit 2024-02-18T01:26:42.993505+0000 mon.a (mon.0) 1934 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: cluster 2024-02-18T01:26:43.723625+0000 mgr.y (mgr.24475) 392 : cluster [DBG] pgmap v493: 236 pgs: 26 creating+peering, 14 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: cluster 2024-02-18T01:26:43.875249+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: audit 2024-02-18T01:26:43.986772+0000 mon.a (mon.0) 1936 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: cluster 2024-02-18T01:26:43.986912+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-18T01:26:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: audit 2024-02-18T01:26:43.991341+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[27669]: audit 2024-02-18T01:26:43.991765+0000 mon.a (mon.0) 1938 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: cluster 2024-02-18T01:26:43.723625+0000 mgr.y (mgr.24475) 392 : cluster [DBG] pgmap v493: 236 pgs: 26 creating+peering, 14 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: cluster 2024-02-18T01:26:43.875249+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: audit 2024-02-18T01:26:43.986772+0000 mon.a (mon.0) 1936 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: cluster 2024-02-18T01:26:43.986912+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: audit 2024-02-18T01:26:43.991341+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:44.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:44 smithi105 bash[19734]: audit 2024-02-18T01:26:43.991765+0000 mon.a (mon.0) 1938 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: cluster 2024-02-18T01:26:43.723625+0000 mgr.y (mgr.24475) 392 : cluster [DBG] pgmap v493: 236 pgs: 26 creating+peering, 14 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: cluster 2024-02-18T01:26:43.875249+0000 mon.a (mon.0) 1935 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: audit 2024-02-18T01:26:43.986772+0000 mon.a (mon.0) 1936 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: cluster 2024-02-18T01:26:43.986912+0000 mon.a (mon.0) 1937 : cluster [DBG] osdmap e278: 8 total, 8 up, 8 in 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: audit 2024-02-18T01:26:43.991341+0000 mon.b (mon.1) 255 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:44 smithi155 bash[20805]: audit 2024-02-18T01:26:43.991765+0000 mon.a (mon.0) 1938 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]: dispatch 2024-02-18T01:26:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[27669]: audit 2024-02-18T01:26:44.990905+0000 mon.a (mon.0) 1939 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T01:26:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[27669]: cluster 2024-02-18T01:26:44.991591+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-18T01:26:46.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[27669]: audit 2024-02-18T01:26:44.992841+0000 mon.b (mon.1) 256 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[27669]: audit 2024-02-18T01:26:44.993304+0000 mon.a (mon.0) 1941 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[27669]: audit 2024-02-18T01:26:44.994377+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:46.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[19734]: audit 2024-02-18T01:26:44.990905+0000 mon.a (mon.0) 1939 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T01:26:46.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[19734]: cluster 2024-02-18T01:26:44.991591+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-18T01:26:46.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[19734]: audit 2024-02-18T01:26:44.992841+0000 mon.b (mon.1) 256 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[19734]: audit 2024-02-18T01:26:44.993304+0000 mon.a (mon.0) 1941 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:45 smithi105 bash[19734]: audit 2024-02-18T01:26:44.994377+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:46.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:45 smithi155 bash[20805]: audit 2024-02-18T01:26:44.990905+0000 mon.a (mon.0) 1939 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_count","val": "8"}]': finished 2024-02-18T01:26:46.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:45 smithi155 bash[20805]: cluster 2024-02-18T01:26:44.991591+0000 mon.a (mon.0) 1940 : cluster [DBG] osdmap e279: 8 total, 8 up, 8 in 2024-02-18T01:26:46.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:45 smithi155 bash[20805]: audit 2024-02-18T01:26:44.992841+0000 mon.b (mon.1) 256 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:45 smithi155 bash[20805]: audit 2024-02-18T01:26:44.993304+0000 mon.a (mon.0) 1941 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:26:46.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:45 smithi155 bash[20805]: audit 2024-02-18T01:26:44.994377+0000 mon.a (mon.0) 1942 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:46] "GET /metrics HTTP/1.1" 200 189644 "" "Prometheus/2.33.4" 2024-02-18T01:26:47.305 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9051 ms) 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg_num = 32 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 0 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 1 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 2 ls 1708219607,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 3 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 4 ls 1708219607,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 5 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 6 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 7 ls 1708219607,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 8 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 9 ls 1708219608,0 2024-02-18T01:26:47.306 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 10 ls 1708219607,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 11 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 12 ls 1708219607,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 13 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 14 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 15 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 16 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 17 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 18 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 19 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 20 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 21 ls 1708219608,0 2024-02-18T01:26:47.307 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 22 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 23 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 24 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 25 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 26 ls 1708219607,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 27 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 28 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 29 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 30 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg 31 ls 1708219608,0 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: pg_num = 32 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6cac518f:::0:head 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:02547ec2:::1:head 2024-02-18T01:26:47.308 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f905c69b:::2:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:cfc208b3:::3:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d83876eb:::4:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b29083e3:::5:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c4fdafeb:::6:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:5c6b0b28:::7:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:bd63b0f1:::8:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e960b815:::9:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:52ea6a34:::10:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:89d3ae78:::11:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:de5d7c5f:::12:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:566253c9:::13:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:62a1935d:::14:head 2024-02-18T01:26:47.309 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:863748b0:::15:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3958e169:::16:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4d4dabf9:::17:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:8391935d:::18:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:28883081:::19:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:69259c59:::20:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4bdb80b7:::21:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a11c5d71:::22:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:271af37b:::23:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:95b121be:::24:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:58d1031b:::25:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:0a050783:::26:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c709704c:::27:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:cbe56eaf:::28:head 2024-02-18T01:26:47.310 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:86b4b162:::29:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:70d89383:::30:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:dd450c7c:::31:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6d5729b1:::32:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c388f3fb:::33:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:56cfea31:::34:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9dbc1bf7:::35:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:40b74ccd:::36:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4d5aaf42:::37:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:920f362c:::38:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6cc53222:::39:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9cad833f:::40:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:1ea84d41:::41:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c4480ef6:::42:head 2024-02-18T01:26:47.311 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a694361e:::43:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d1bd33e9:::44:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ddc2cd5d:::45:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:2b782207:::46:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7b187fca:::47:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:90ecdf6f:::48:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a5ed95fe:::49:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ea0eaa55:::50:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f33ef17b:::51:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a0d1b2f6:::52:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:60c5229e:::53:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:edcbc575:::54:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:102cf253:::55:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:efb7fb0b:::56:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:50d0a326:::57:head 2024-02-18T01:26:47.312 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d4dc5daf:::58:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3a130462:::59:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ec87ed71:::60:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d5bc9454:::61:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3ddfe313:::62:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7c2816b9:::63:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:47e00e4d:::64:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c6410c18:::65:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b48ed237:::66:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:cd63ad31:::67:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b179e92b:::68:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:0d9f741a:::69:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6d3352ae:::70:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c6d5c19e:::71:head 2024-02-18T01:26:47.313 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:bc4729c3:::72:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:77e930b9:::73:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:0abeecfd:::74:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b7c37e15:::75:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b6378398:::76:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:02bd68de:::77:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:cc795d2d:::78:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:630d4fea:::79:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e0d29ef5:::80:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:fd6f13d2:::81:head 2024-02-18T01:26:47.314 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:606461d5:::82:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:eadbdc43:::83:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:8761d0bb:::84:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9ef0186f:::85:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e0d41294:::86:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:961de695:::87:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:1423148f:::88:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:633a8fa2:::89:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a8653809:::90:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3dac8b33:::91:head 2024-02-18T01:26:47.315 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:35aad435:::92:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f6dcc343:::93:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:dbbdad87:::94:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:1cb48ce0:::95:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:03cd461c:::96:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:17a4ea99:::97:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9993c9a7:::98:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6394211c:::99:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:94c7ae57:::100:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6fdee5bb:::101:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9a477fd1:::102:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:eb850916:::103:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:affc56b9:::104:head 2024-02-18T01:26:47.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b42dc814:::105:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f319f8f0:::106:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9a40b9de:::107:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:8b524f28:::108:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e3de589f:::109:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:90f90a5b:::110:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a7b4f1d7:::111:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:af51766e:::112:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b6f90bd1:::113:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e0261208:::114:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c9569ef7:::115:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:61bebe50:::116:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:fe93412b:::117:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d3d38bee:::118:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3100ba0c:::119:head 2024-02-18T01:26:47.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d0560ada:::120:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f0ea8b35:::121:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:766f231a:::122:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a07a2582:::123:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:bd7c6b3a:::124:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:fb2ddaff:::125:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4408e1fe:::126:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ee1df7a7:::127:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c3002909:::128:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4f48ffa9:::129:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:edf38733:::130:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c08425c0:::131:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:5f902d98:::132:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:41ea2c93:::133:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:813cee13:::134:head 2024-02-18T01:26:47.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:0131818d:::135:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:26ba5a85:::136:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:381b8a5a:::137:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:28797e47:::138:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:bfca7f22:::139:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:36807075:::140:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:80b03975:::141:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:5c15709b:::142:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f39ea15e:::143:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ea992956:::144:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:48887b1c:::145:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:9f24a9dd:::146:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:987f100b:::147:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d2dd3581:::148:head 2024-02-18T01:26:47.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7fed1808:::149:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c80b70e9:::150:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:85ed90f9:::151:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:36428b24:::152:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d044c34a:::153:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7c18bf58:::154:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d1c21232:::155:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a7a3c575:::156:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:87da0633:::157:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d5ac3822:::158:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3f20522d:::159:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6ca26563:::160:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:532ce135:::161:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:c78863e6:::162:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7462ddf6:::163:head 2024-02-18T01:26:47.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:5d165639:::164:head 2024-02-18T01:26:47.321 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f43765fc:::165:head 2024-02-18T01:26:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: cluster 2024-02-18T01:26:45.724378+0000 mgr.y (mgr.24475) 393 : cluster [DBG] pgmap v496: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: audit 2024-02-18T01:26:45.999714+0000 mon.a (mon.0) 1943 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: audit 2024-02-18T01:26:45.999810+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: cluster 2024-02-18T01:26:45.999880+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: audit 2024-02-18T01:26:46.000457+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: audit 2024-02-18T01:26:46.003558+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[27669]: audit 2024-02-18T01:26:46.003612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: cluster 2024-02-18T01:26:45.724378+0000 mgr.y (mgr.24475) 393 : cluster [DBG] pgmap v496: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: audit 2024-02-18T01:26:45.999714+0000 mon.a (mon.0) 1943 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: audit 2024-02-18T01:26:45.999810+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: cluster 2024-02-18T01:26:45.999880+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: audit 2024-02-18T01:26:46.000457+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: audit 2024-02-18T01:26:46.003558+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:47 smithi105 bash[19734]: audit 2024-02-18T01:26:46.003612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: cluster 2024-02-18T01:26:45.724378+0000 mgr.y (mgr.24475) 393 : cluster [DBG] pgmap v496: 228 pgs: 26 creating+peering, 6 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: audit 2024-02-18T01:26:45.999714+0000 mon.a (mon.0) 1943 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: audit 2024-02-18T01:26:45.999810+0000 mon.a (mon.0) 1944 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: cluster 2024-02-18T01:26:45.999880+0000 mon.a (mon.0) 1945 : cluster [DBG] osdmap e280: 8 total, 8 up, 8 in 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: audit 2024-02-18T01:26:46.000457+0000 mon.a (mon.0) 1946 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]: dispatch 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: audit 2024-02-18T01:26:46.003558+0000 mon.b (mon.1) 257 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[20805]: audit 2024-02-18T01:26:46.003612+0000 mon.a (mon.0) 1947 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-02-18T01:26:48.019 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.002133+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.002244+0000 mon.a (mon.0) 1949 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T01:26:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: cluster 2024-02-18T01:26:47.002344+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.026254+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.027475+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.027920+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.304443+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.595048+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.595378+0000 mon.a (mon.0) 1954 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.596936+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[19734]: audit 2024-02-18T01:26:47.597229+0000 mon.a (mon.0) 1955 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.002133+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:48.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.002244+0000 mon.a (mon.0) 1949 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: cluster 2024-02-18T01:26:47.002344+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.026254+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.027475+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.027920+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.304443+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.595048+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.595378+0000 mon.a (mon.0) 1954 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.596936+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:48.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:48 smithi105 bash[27669]: audit 2024-02-18T01:26:47.597229+0000 mon.a (mon.0) 1955 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.002133+0000 mon.a (mon.0) 1948 : audit [INF] from='client.? v1:172.21.15.105:0/2920474300' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi105-61657-54"}]': finished 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.002244+0000 mon.a (mon.0) 1949 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-41","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: cluster 2024-02-18T01:26:47.002344+0000 mon.a (mon.0) 1950 : cluster [DBG] osdmap e281: 8 total, 8 up, 8 in 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.026254+0000 mon.a (mon.0) 1951 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.027475+0000 mon.a (mon.0) 1952 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.027920+0000 mon.a (mon.0) 1953 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.304443+0000 mon.b (mon.1) 258 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi105-61833-41","var": "pg_num","format": "json"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.595048+0000 mon.b (mon.1) 259 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.595378+0000 mon.a (mon.0) 1954 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.596936+0000 mon.b (mon.1) 260 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:48.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:48 smithi155 bash[20805]: audit 2024-02-18T01:26:47.597229+0000 mon.a (mon.0) 1955 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]: dispatch 2024-02-18T01:26:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: cluster 2024-02-18T01:26:47.726676+0000 mgr.y (mgr.24475) 394 : cluster [DBG] pgmap v499: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-18T01:26:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: audit 2024-02-18T01:26:47.761058+0000 mgr.y (mgr.24475) 395 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: audit 2024-02-18T01:26:48.031523+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: audit 2024-02-18T01:26:48.031650+0000 mon.a (mon.0) 1957 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]': finished 2024-02-18T01:26:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: cluster 2024-02-18T01:26:48.031758+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[27669]: audit 2024-02-18T01:26:48.033934+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: cluster 2024-02-18T01:26:47.726676+0000 mgr.y (mgr.24475) 394 : cluster [DBG] pgmap v499: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: audit 2024-02-18T01:26:47.761058+0000 mgr.y (mgr.24475) 395 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: audit 2024-02-18T01:26:48.031523+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: audit 2024-02-18T01:26:48.031650+0000 mon.a (mon.0) 1957 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]': finished 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: cluster 2024-02-18T01:26:48.031758+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-18T01:26:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:49 smithi105 bash[19734]: audit 2024-02-18T01:26:48.033934+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: cluster 2024-02-18T01:26:47.726676+0000 mgr.y (mgr.24475) 394 : cluster [DBG] pgmap v499: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 50 KiB/s wr, 51 op/s 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: audit 2024-02-18T01:26:47.761058+0000 mgr.y (mgr.24475) 395 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: audit 2024-02-18T01:26:48.031523+0000 mon.a (mon.0) 1956 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi105-61657-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: audit 2024-02-18T01:26:48.031650+0000 mon.a (mon.0) 1957 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-41"}]': finished 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: cluster 2024-02-18T01:26:48.031758+0000 mon.a (mon.0) 1958 : cluster [DBG] osdmap e282: 8 total, 8 up, 8 in 2024-02-18T01:26:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:49 smithi155 bash[20805]: audit 2024-02-18T01:26:48.033934+0000 mon.a (mon.0) 1959 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:50 smithi105 bash[19734]: cluster 2024-02-18T01:26:49.463191+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-18T01:26:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:50 smithi105 bash[19734]: cluster 2024-02-18T01:26:49.727414+0000 mgr.y (mgr.24475) 396 : cluster [DBG] pgmap v502: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:50 smithi105 bash[27669]: cluster 2024-02-18T01:26:49.463191+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-18T01:26:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:50 smithi105 bash[27669]: cluster 2024-02-18T01:26:49.727414+0000 mgr.y (mgr.24475) 396 : cluster [DBG] pgmap v502: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:50.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:50 smithi155 bash[20805]: cluster 2024-02-18T01:26:49.463191+0000 mon.a (mon.0) 1960 : cluster [DBG] osdmap e283: 8 total, 8 up, 8 in 2024-02-18T01:26:50.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:50 smithi155 bash[20805]: cluster 2024-02-18T01:26:49.727414+0000 mgr.y (mgr.24475) 396 : cluster [DBG] pgmap v502: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[27669]: audit 2024-02-18T01:26:50.049663+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[27669]: cluster 2024-02-18T01:26:50.049881+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-18T01:26:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[27669]: audit 2024-02-18T01:26:50.055982+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:51.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[27669]: audit 2024-02-18T01:26:50.056217+0000 mon.a (mon.0) 1963 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:51.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[19734]: audit 2024-02-18T01:26:50.049663+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:51.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[19734]: cluster 2024-02-18T01:26:50.049881+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-18T01:26:51.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[19734]: audit 2024-02-18T01:26:50.055982+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:51.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:51 smithi105 bash[19734]: audit 2024-02-18T01:26:50.056217+0000 mon.a (mon.0) 1963 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:51.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:51 smithi155 bash[20805]: audit 2024-02-18T01:26:50.049663+0000 mon.a (mon.0) 1961 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi105-61657-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:51.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:51 smithi155 bash[20805]: cluster 2024-02-18T01:26:50.049881+0000 mon.a (mon.0) 1962 : cluster [DBG] osdmap e284: 8 total, 8 up, 8 in 2024-02-18T01:26:51.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:51 smithi155 bash[20805]: audit 2024-02-18T01:26:50.055982+0000 mon.b (mon.1) 261 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:51.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:51 smithi155 bash[20805]: audit 2024-02-18T01:26:50.056217+0000 mon.a (mon.0) 1963 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:26:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:52 smithi155 bash[20805]: audit 2024-02-18T01:26:51.058561+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:52 smithi155 bash[20805]: cluster 2024-02-18T01:26:51.059141+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:52 smithi155 bash[20805]: audit 2024-02-18T01:26:51.061324+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:52 smithi155 bash[20805]: audit 2024-02-18T01:26:51.069160+0000 mon.a (mon.0) 1966 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:52 smithi155 bash[20805]: cluster 2024-02-18T01:26:51.728179+0000 mgr.y (mgr.24475) 397 : cluster [DBG] pgmap v505: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[27669]: audit 2024-02-18T01:26:51.058561+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[27669]: cluster 2024-02-18T01:26:51.059141+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-18T01:26:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[27669]: audit 2024-02-18T01:26:51.061324+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[27669]: audit 2024-02-18T01:26:51.069160+0000 mon.a (mon.0) 1966 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[27669]: cluster 2024-02-18T01:26:51.728179+0000 mgr.y (mgr.24475) 397 : cluster [DBG] pgmap v505: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:52.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[19734]: audit 2024-02-18T01:26:51.058561+0000 mon.a (mon.0) 1964 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:26:52.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[19734]: cluster 2024-02-18T01:26:51.059141+0000 mon.a (mon.0) 1965 : cluster [DBG] osdmap e285: 8 total, 8 up, 8 in 2024-02-18T01:26:52.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[19734]: audit 2024-02-18T01:26:51.061324+0000 mon.b (mon.1) 262 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[19734]: audit 2024-02-18T01:26:51.069160+0000 mon.a (mon.0) 1966 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:26:52.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:52 smithi105 bash[19734]: cluster 2024-02-18T01:26:51.728179+0000 mgr.y (mgr.24475) 397 : cluster [DBG] pgmap v505: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:26:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:52.063351+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: cluster 2024-02-18T01:26:52.063567+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:52.065230+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:52.068760+0000 mon.a (mon.0) 1969 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:52.069713+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:53.063885+0000 mon.a (mon.0) 1971 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:53.064079+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: cluster 2024-02-18T01:26:53.064173+0000 mon.a (mon.0) 1973 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:53.065491+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:53.066956+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[19734]: audit 2024-02-18T01:26:53.067495+0000 mon.a (mon.0) 1975 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:53.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:52.063351+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: cluster 2024-02-18T01:26:52.063567+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:52.065230+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:52.068760+0000 mon.a (mon.0) 1969 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:52.069713+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:53.063885+0000 mon.a (mon.0) 1971 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:53.064079+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: cluster 2024-02-18T01:26:53.064173+0000 mon.a (mon.0) 1973 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:53.065491+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:53.066956+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:53.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:53 smithi105 bash[27669]: audit 2024-02-18T01:26:53.067495+0000 mon.a (mon.0) 1975 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:52.063351+0000 mon.a (mon.0) 1967 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: cluster 2024-02-18T01:26:52.063567+0000 mon.a (mon.0) 1968 : cluster [DBG] osdmap e286: 8 total, 8 up, 8 in 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:52.065230+0000 mon.b (mon.1) 263 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:52.068760+0000 mon.a (mon.0) 1969 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:52.069713+0000 mon.a (mon.0) 1970 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:53.063885+0000 mon.a (mon.0) 1971 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:53.064079+0000 mon.a (mon.0) 1972 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:53.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: cluster 2024-02-18T01:26:53.064173+0000 mon.a (mon.0) 1973 : cluster [DBG] osdmap e287: 8 total, 8 up, 8 in 2024-02-18T01:26:53.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:53.065491+0000 mon.a (mon.0) 1974 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]: dispatch 2024-02-18T01:26:53.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:53.066956+0000 mon.b (mon.1) 264 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:53.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:53 smithi155 bash[20805]: audit 2024-02-18T01:26:53.067495+0000 mon.a (mon.0) 1975 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:26:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: cluster 2024-02-18T01:26:53.728978+0000 mgr.y (mgr.24475) 398 : cluster [DBG] pgmap v508: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: audit 2024-02-18T01:26:54.067359+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: audit 2024-02-18T01:26:54.067441+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:26:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: cluster 2024-02-18T01:26:54.067527+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-18T01:26:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: audit 2024-02-18T01:26:54.071267+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[27669]: audit 2024-02-18T01:26:54.071456+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: cluster 2024-02-18T01:26:53.728978+0000 mgr.y (mgr.24475) 398 : cluster [DBG] pgmap v508: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: audit 2024-02-18T01:26:54.067359+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: audit 2024-02-18T01:26:54.067441+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: cluster 2024-02-18T01:26:54.067527+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: audit 2024-02-18T01:26:54.071267+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:54 smithi105 bash[19734]: audit 2024-02-18T01:26:54.071456+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: cluster 2024-02-18T01:26:53.728978+0000 mgr.y (mgr.24475) 398 : cluster [DBG] pgmap v508: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: audit 2024-02-18T01:26:54.067359+0000 mon.a (mon.0) 1976 : audit [INF] from='client.? v1:172.21.15.105:0/2438921747' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi105-61657-55"}]': finished 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: audit 2024-02-18T01:26:54.067441+0000 mon.a (mon.0) 1977 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: cluster 2024-02-18T01:26:54.067527+0000 mon.a (mon.0) 1978 : cluster [DBG] osdmap e288: 8 total, 8 up, 8 in 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: audit 2024-02-18T01:26:54.071267+0000 mon.b (mon.1) 265 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:54.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:54 smithi155 bash[20805]: audit 2024-02-18T01:26:54.071456+0000 mon.a (mon.0) 1979 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:26:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:54.092190+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:54.093751+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:54.094211+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:55.071695+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:55.071953+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: cluster 2024-02-18T01:26:55.072073+0000 mon.a (mon.0) 1985 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:55.073977+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:55.074188+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[19734]: audit 2024-02-18T01:26:55.074506+0000 mon.a (mon.0) 1987 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:54.092190+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:54.093751+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:54.094211+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:55.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:55.071695+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:26:55.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:55.071953+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:55.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: cluster 2024-02-18T01:26:55.072073+0000 mon.a (mon.0) 1985 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-18T01:26:55.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:55.073977+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:55.074188+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:55.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:55 smithi105 bash[27669]: audit 2024-02-18T01:26:55.074506+0000 mon.a (mon.0) 1987 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:54.092190+0000 mon.a (mon.0) 1980 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:54.093751+0000 mon.a (mon.0) 1981 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:54.094211+0000 mon.a (mon.0) 1982 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:55.071695+0000 mon.a (mon.0) 1983 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:55.071953+0000 mon.a (mon.0) 1984 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi105-61657-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: cluster 2024-02-18T01:26:55.072073+0000 mon.a (mon.0) 1985 : cluster [DBG] osdmap e289: 8 total, 8 up, 8 in 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:55.073977+0000 mon.a (mon.0) 1986 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:55.074188+0000 mon.b (mon.1) 266 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:55 smithi155 bash[20805]: audit 2024-02-18T01:26:55.074506+0000 mon.a (mon.0) 1987 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:26:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:56 smithi105 bash[19734]: cluster 2024-02-18T01:26:55.729834+0000 mgr.y (mgr.24475) 399 : cluster [DBG] pgmap v511: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:56 smithi105 bash[27669]: cluster 2024-02-18T01:26:55.729834+0000 mgr.y (mgr.24475) 399 : cluster [DBG] pgmap v511: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:56.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:26:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:26:56] "GET /metrics HTTP/1.1" 200 189879 "" "Prometheus/2.33.4" 2024-02-18T01:26:56.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:56 smithi155 bash[20805]: cluster 2024-02-18T01:26:55.729834+0000 mgr.y (mgr.24475) 399 : cluster [DBG] pgmap v511: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:26:57.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:57 smithi155 bash[20805]: audit 2024-02-18T01:26:56.501792+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:26:57.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:57 smithi155 bash[20805]: cluster 2024-02-18T01:26:56.502001+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-18T01:26:57.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:57 smithi155 bash[20805]: audit 2024-02-18T01:26:57.078393+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:26:57.769 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:57 smithi155 bash[20805]: cluster 2024-02-18T01:26:57.078652+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-18T01:26:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[27669]: audit 2024-02-18T01:26:56.501792+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:26:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[27669]: cluster 2024-02-18T01:26:56.502001+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-18T01:26:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[27669]: audit 2024-02-18T01:26:57.078393+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:26:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[27669]: cluster 2024-02-18T01:26:57.078652+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-18T01:26:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[19734]: audit 2024-02-18T01:26:56.501792+0000 mon.a (mon.0) 1988 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-43","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:26:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[19734]: cluster 2024-02-18T01:26:56.502001+0000 mon.a (mon.0) 1989 : cluster [DBG] osdmap e290: 8 total, 8 up, 8 in 2024-02-18T01:26:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[19734]: audit 2024-02-18T01:26:57.078393+0000 mon.a (mon.0) 1990 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi105-61657-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:26:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:57 smithi105 bash[19734]: cluster 2024-02-18T01:26:57.078652+0000 mon.a (mon.0) 1991 : cluster [DBG] osdmap e291: 8 total, 8 up, 8 in 2024-02-18T01:26:58.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:26:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:26:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[27669]: cluster 2024-02-18T01:26:57.732122+0000 mgr.y (mgr.24475) 400 : cluster [DBG] pgmap v514: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[27669]: audit 2024-02-18T01:26:57.767135+0000 mgr.y (mgr.24475) 401 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[27669]: cluster 2024-02-18T01:26:58.080316+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-18T01:26:58.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[19734]: cluster 2024-02-18T01:26:57.732122+0000 mgr.y (mgr.24475) 400 : cluster [DBG] pgmap v514: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:58.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[19734]: audit 2024-02-18T01:26:57.767135+0000 mgr.y (mgr.24475) 401 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:58.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:26:58 smithi105 bash[19734]: cluster 2024-02-18T01:26:58.080316+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-18T01:26:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:58 smithi155 bash[20805]: cluster 2024-02-18T01:26:57.732122+0000 mgr.y (mgr.24475) 400 : cluster [DBG] pgmap v514: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:26:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:58 smithi155 bash[20805]: audit 2024-02-18T01:26:57.767135+0000 mgr.y (mgr.24475) 401 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:26:58.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:26:58 smithi155 bash[20805]: cluster 2024-02-18T01:26:58.080316+0000 mon.a (mon.0) 1992 : cluster [DBG] osdmap e292: 8 total, 8 up, 8 in 2024-02-18T01:27:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[19734]: cluster 2024-02-18T01:26:59.086409+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-18T01:27:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[19734]: audit 2024-02-18T01:26:59.087521+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[19734]: cluster 2024-02-18T01:26:59.732928+0000 mgr.y (mgr.24475) 402 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[27669]: cluster 2024-02-18T01:26:59.086409+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-18T01:27:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[27669]: audit 2024-02-18T01:26:59.087521+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:00 smithi105 bash[27669]: cluster 2024-02-18T01:26:59.732928+0000 mgr.y (mgr.24475) 402 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:00 smithi155 bash[20805]: cluster 2024-02-18T01:26:59.086409+0000 mon.a (mon.0) 1993 : cluster [DBG] osdmap e293: 8 total, 8 up, 8 in 2024-02-18T01:27:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:00 smithi155 bash[20805]: audit 2024-02-18T01:26:59.087521+0000 mon.a (mon.0) 1994 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:00.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:00 smithi155 bash[20805]: cluster 2024-02-18T01:26:59.732928+0000 mgr.y (mgr.24475) 402 : cluster [DBG] pgmap v517: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[27669]: audit 2024-02-18T01:27:00.087165+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[27669]: cluster 2024-02-18T01:27:00.087309+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-18T01:27:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[27669]: audit 2024-02-18T01:27:00.090729+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[27669]: audit 2024-02-18T01:27:01.088287+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[27669]: cluster 2024-02-18T01:27:01.088396+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[19734]: audit 2024-02-18T01:27:00.087165+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[19734]: cluster 2024-02-18T01:27:00.087309+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[19734]: audit 2024-02-18T01:27:00.090729+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[19734]: audit 2024-02-18T01:27:01.088287+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:01 smithi105 bash[19734]: cluster 2024-02-18T01:27:01.088396+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-18T01:27:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[20805]: audit 2024-02-18T01:27:00.087165+0000 mon.a (mon.0) 1995 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[20805]: cluster 2024-02-18T01:27:00.087309+0000 mon.a (mon.0) 1996 : cluster [DBG] osdmap e294: 8 total, 8 up, 8 in 2024-02-18T01:27:01.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[20805]: audit 2024-02-18T01:27:00.090729+0000 mon.a (mon.0) 1997 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]: dispatch 2024-02-18T01:27:01.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[20805]: audit 2024-02-18T01:27:01.088287+0000 mon.a (mon.0) 1998 : audit [INF] from='client.? v1:172.21.15.105:0/3935164096' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi105-61657-56"}]': finished 2024-02-18T01:27:01.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[20805]: cluster 2024-02-18T01:27:01.088396+0000 mon.a (mon.0) 1999 : cluster [DBG] osdmap e295: 8 total, 8 up, 8 in 2024-02-18T01:27:02.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.110533+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.110616+0000 mon.a (mon.0) 2000 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.112373+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.112460+0000 mon.a (mon.0) 2001 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.113352+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: audit 2024-02-18T01:27:01.113402+0000 mon.a (mon.0) 2002 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:02 smithi155 bash[20805]: cluster 2024-02-18T01:27:01.733788+0000 mgr.y (mgr.24475) 403 : cluster [DBG] pgmap v520: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:02.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.110533+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.110616+0000 mon.a (mon.0) 2000 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.112373+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.112460+0000 mon.a (mon.0) 2001 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.113352+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: audit 2024-02-18T01:27:01.113402+0000 mon.a (mon.0) 2002 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[27669]: cluster 2024-02-18T01:27:01.733788+0000 mgr.y (mgr.24475) 403 : cluster [DBG] pgmap v520: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.110533+0000 mon.b (mon.1) 267 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.110616+0000 mon.a (mon.0) 2000 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.112373+0000 mon.b (mon.1) 268 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.112460+0000 mon.a (mon.0) 2001 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.113352+0000 mon.b (mon.1) 269 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: audit 2024-02-18T01:27:01.113402+0000 mon.a (mon.0) 2002 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:02.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:02 smithi105 bash[19734]: cluster 2024-02-18T01:27:01.733788+0000 mgr.y (mgr.24475) 403 : cluster [DBG] pgmap v520: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[19734]: audit 2024-02-18T01:27:02.112160+0000 mon.a (mon.0) 2003 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[19734]: cluster 2024-02-18T01:27:02.112263+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-18T01:27:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[19734]: audit 2024-02-18T01:27:02.114616+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[19734]: audit 2024-02-18T01:27:02.114866+0000 mon.a (mon.0) 2005 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[27669]: audit 2024-02-18T01:27:02.112160+0000 mon.a (mon.0) 2003 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[27669]: cluster 2024-02-18T01:27:02.112263+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-18T01:27:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[27669]: audit 2024-02-18T01:27:02.114616+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:03 smithi105 bash[27669]: audit 2024-02-18T01:27:02.114866+0000 mon.a (mon.0) 2005 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:03 smithi155 bash[20805]: audit 2024-02-18T01:27:02.112160+0000 mon.a (mon.0) 2003 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi105-61657-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:03 smithi155 bash[20805]: cluster 2024-02-18T01:27:02.112263+0000 mon.a (mon.0) 2004 : cluster [DBG] osdmap e296: 8 total, 8 up, 8 in 2024-02-18T01:27:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:03 smithi155 bash[20805]: audit 2024-02-18T01:27:02.114616+0000 mon.b (mon.1) 270 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:03 smithi155 bash[20805]: audit 2024-02-18T01:27:02.114866+0000 mon.a (mon.0) 2005 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:04.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[19734]: cluster 2024-02-18T01:27:03.551911+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-18T01:27:04.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[19734]: cluster 2024-02-18T01:27:03.734705+0000 mgr.y (mgr.24475) 404 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:04.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[19734]: audit 2024-02-18T01:27:04.118602+0000 mon.a (mon.0) 2007 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:04.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[19734]: cluster 2024-02-18T01:27:04.118723+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-18T01:27:04.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[27669]: cluster 2024-02-18T01:27:03.551911+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-18T01:27:04.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[27669]: cluster 2024-02-18T01:27:03.734705+0000 mgr.y (mgr.24475) 404 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:04.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[27669]: audit 2024-02-18T01:27:04.118602+0000 mon.a (mon.0) 2007 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:04.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:04 smithi105 bash[27669]: cluster 2024-02-18T01:27:04.118723+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-18T01:27:04.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:04 smithi155 bash[20805]: cluster 2024-02-18T01:27:03.551911+0000 mon.a (mon.0) 2006 : cluster [DBG] osdmap e297: 8 total, 8 up, 8 in 2024-02-18T01:27:04.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:04 smithi155 bash[20805]: cluster 2024-02-18T01:27:03.734705+0000 mgr.y (mgr.24475) 404 : cluster [DBG] pgmap v523: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:04.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:04 smithi155 bash[20805]: audit 2024-02-18T01:27:04.118602+0000 mon.a (mon.0) 2007 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi105-61657-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:04.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:04 smithi155 bash[20805]: cluster 2024-02-18T01:27:04.118723+0000 mon.a (mon.0) 2008 : cluster [DBG] osdmap e298: 8 total, 8 up, 8 in 2024-02-18T01:27:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:06 smithi105 bash[19734]: cluster 2024-02-18T01:27:05.128583+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-18T01:27:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:06 smithi105 bash[19734]: cluster 2024-02-18T01:27:05.735599+0000 mgr.y (mgr.24475) 405 : cluster [DBG] pgmap v526: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:06 smithi105 bash[27669]: cluster 2024-02-18T01:27:05.128583+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-18T01:27:06.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:06 smithi105 bash[27669]: cluster 2024-02-18T01:27:05.735599+0000 mgr.y (mgr.24475) 405 : cluster [DBG] pgmap v526: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:06 smithi155 bash[20805]: cluster 2024-02-18T01:27:05.128583+0000 mon.a (mon.0) 2009 : cluster [DBG] osdmap e299: 8 total, 8 up, 8 in 2024-02-18T01:27:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:06 smithi155 bash[20805]: cluster 2024-02-18T01:27:05.735599+0000 mgr.y (mgr.24475) 405 : cluster [DBG] pgmap v526: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:06] "GET /metrics HTTP/1.1" 200 187513 "" "Prometheus/2.33.4" 2024-02-18T01:27:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[19734]: cluster 2024-02-18T01:27:06.134759+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-18T01:27:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[19734]: audit 2024-02-18T01:27:06.137931+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[19734]: audit 2024-02-18T01:27:06.138417+0000 mon.a (mon.0) 2011 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[27669]: cluster 2024-02-18T01:27:06.134759+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-18T01:27:07.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[27669]: audit 2024-02-18T01:27:06.137931+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:07.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:07 smithi105 bash[27669]: audit 2024-02-18T01:27:06.138417+0000 mon.a (mon.0) 2011 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:07 smithi155 bash[20805]: cluster 2024-02-18T01:27:06.134759+0000 mon.a (mon.0) 2010 : cluster [DBG] osdmap e300: 8 total, 8 up, 8 in 2024-02-18T01:27:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:07 smithi155 bash[20805]: audit 2024-02-18T01:27:06.137931+0000 mon.b (mon.1) 271 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:07.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:07 smithi155 bash[20805]: audit 2024-02-18T01:27:06.138417+0000 mon.a (mon.0) 2011 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.145 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: audit 2024-02-18T01:27:07.143945+0000 mon.a (mon.0) 2012 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: cluster 2024-02-18T01:27:07.144014+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-18T01:27:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: audit 2024-02-18T01:27:07.148607+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: audit 2024-02-18T01:27:07.150050+0000 mon.a (mon.0) 2014 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: cluster 2024-02-18T01:27:07.737811+0000 mgr.y (mgr.24475) 406 : cluster [DBG] pgmap v529: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[19734]: audit 2024-02-18T01:27:07.778458+0000 mgr.y (mgr.24475) 407 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: audit 2024-02-18T01:27:07.143945+0000 mon.a (mon.0) 2012 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: cluster 2024-02-18T01:27:07.144014+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: audit 2024-02-18T01:27:07.148607+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: audit 2024-02-18T01:27:07.150050+0000 mon.a (mon.0) 2014 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: cluster 2024-02-18T01:27:07.737811+0000 mgr.y (mgr.24475) 406 : cluster [DBG] pgmap v529: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:08 smithi105 bash[27669]: audit 2024-02-18T01:27:07.778458+0000 mgr.y (mgr.24475) 407 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: audit 2024-02-18T01:27:07.143945+0000 mon.a (mon.0) 2012 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: cluster 2024-02-18T01:27:07.144014+0000 mon.a (mon.0) 2013 : cluster [DBG] osdmap e301: 8 total, 8 up, 8 in 2024-02-18T01:27:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: audit 2024-02-18T01:27:07.148607+0000 mon.b (mon.1) 272 : audit [INF] from='client.? v1:172.21.15.105:0/1870023738' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: audit 2024-02-18T01:27:07.150050+0000 mon.a (mon.0) 2014 : audit [INF] from='client.48919 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]: dispatch 2024-02-18T01:27:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: cluster 2024-02-18T01:27:07.737811+0000 mgr.y (mgr.24475) 406 : cluster [DBG] pgmap v529: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:08 smithi155 bash[20805]: audit 2024-02-18T01:27:07.778458+0000 mgr.y (mgr.24475) 407 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.156652+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:09.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: cluster 2024-02-18T01:27:08.156790+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-18T01:27:09.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.177797+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.179955+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.180428+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:09.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.587099+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.587516+0000 mon.a (mon.0) 2020 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.588861+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:09.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[19734]: audit 2024-02-18T01:27:08.589323+0000 mon.a (mon.0) 2021 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:09.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.156652+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: cluster 2024-02-18T01:27:08.156790+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.177797+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.179955+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.180428+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.587099+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.587516+0000 mon.a (mon.0) 2020 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.588861+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:09.494 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:09 smithi105 bash[27669]: audit 2024-02-18T01:27:08.589323+0000 mon.a (mon.0) 2021 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:09.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.156652+0000 mon.a (mon.0) 2015 : audit [INF] from='client.48919 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi105-61657-57"}]': finished 2024-02-18T01:27:09.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: cluster 2024-02-18T01:27:08.156790+0000 mon.a (mon.0) 2016 : cluster [DBG] osdmap e302: 8 total, 8 up, 8 in 2024-02-18T01:27:09.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.177797+0000 mon.a (mon.0) 2017 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.179955+0000 mon.a (mon.0) 2018 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.180428+0000 mon.a (mon.0) 2019 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.587099+0000 mon.b (mon.1) 273 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.587516+0000 mon.a (mon.0) 2020 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.588861+0000 mon.b (mon.1) 274 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:09.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:09 smithi155 bash[20805]: audit 2024-02-18T01:27:08.589323+0000 mon.a (mon.0) 2021 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]: dispatch 2024-02-18T01:27:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[27669]: audit 2024-02-18T01:27:09.170978+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:10.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[27669]: audit 2024-02-18T01:27:09.171160+0000 mon.a (mon.0) 2023 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]': finished 2024-02-18T01:27:10.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[27669]: cluster 2024-02-18T01:27:09.171305+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-18T01:27:10.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[27669]: audit 2024-02-18T01:27:09.177619+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:10.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[27669]: cluster 2024-02-18T01:27:09.738607+0000 mgr.y (mgr.24475) 408 : cluster [DBG] pgmap v532: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[19734]: audit 2024-02-18T01:27:09.170978+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[19734]: audit 2024-02-18T01:27:09.171160+0000 mon.a (mon.0) 2023 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]': finished 2024-02-18T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[19734]: cluster 2024-02-18T01:27:09.171305+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-18T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[19734]: audit 2024-02-18T01:27:09.177619+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:10.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:10 smithi105 bash[19734]: cluster 2024-02-18T01:27:09.738607+0000 mgr.y (mgr.24475) 408 : cluster [DBG] pgmap v532: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:10 smithi155 bash[20805]: audit 2024-02-18T01:27:09.170978+0000 mon.a (mon.0) 2022 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:10 smithi155 bash[20805]: audit 2024-02-18T01:27:09.171160+0000 mon.a (mon.0) 2023 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-43"}]': finished 2024-02-18T01:27:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:10 smithi155 bash[20805]: cluster 2024-02-18T01:27:09.171305+0000 mon.a (mon.0) 2024 : cluster [DBG] osdmap e303: 8 total, 8 up, 8 in 2024-02-18T01:27:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:10 smithi155 bash[20805]: audit 2024-02-18T01:27:09.177619+0000 mon.a (mon.0) 2025 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:10.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:10 smithi155 bash[20805]: cluster 2024-02-18T01:27:09.738607+0000 mgr.y (mgr.24475) 408 : cluster [DBG] pgmap v532: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-02-18T01:27:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:11 smithi105 bash[19734]: cluster 2024-02-18T01:27:10.603658+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-18T01:27:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:11 smithi105 bash[27669]: cluster 2024-02-18T01:27:10.603658+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-18T01:27:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:11 smithi155 bash[20805]: cluster 2024-02-18T01:27:10.603658+0000 mon.a (mon.0) 2026 : cluster [DBG] osdmap e304: 8 total, 8 up, 8 in 2024-02-18T01:27:12.208 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:12.208 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: audit 2024-02-18T01:27:11.189956+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: cluster 2024-02-18T01:27:11.190104+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: audit 2024-02-18T01:27:11.193566+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: audit 2024-02-18T01:27:11.194931+0000 mon.a (mon.0) 2029 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: cluster 2024-02-18T01:27:11.739438+0000 mgr.y (mgr.24475) 409 : cluster [DBG] pgmap v535: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: audit 2024-02-18T01:27:12.189306+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:12.209 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:12 smithi155 bash[20805]: cluster 2024-02-18T01:27:12.189686+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-18T01:27:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: audit 2024-02-18T01:27:11.189956+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: cluster 2024-02-18T01:27:11.190104+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-18T01:27:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: audit 2024-02-18T01:27:11.193566+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: audit 2024-02-18T01:27:11.194931+0000 mon.a (mon.0) 2029 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: cluster 2024-02-18T01:27:11.739438+0000 mgr.y (mgr.24475) 409 : cluster [DBG] pgmap v535: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: audit 2024-02-18T01:27:12.189306+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[19734]: cluster 2024-02-18T01:27:12.189686+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: audit 2024-02-18T01:27:11.189956+0000 mon.a (mon.0) 2027 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi105-61657-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: cluster 2024-02-18T01:27:11.190104+0000 mon.a (mon.0) 2028 : cluster [DBG] osdmap e305: 8 total, 8 up, 8 in 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: audit 2024-02-18T01:27:11.193566+0000 mon.b (mon.1) 275 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: audit 2024-02-18T01:27:11.194931+0000 mon.a (mon.0) 2029 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: cluster 2024-02-18T01:27:11.739438+0000 mgr.y (mgr.24475) 409 : cluster [DBG] pgmap v535: 236 pgs: 40 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:12.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: audit 2024-02-18T01:27:12.189306+0000 mon.a (mon.0) 2030 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:12.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:12 smithi105 bash[27669]: cluster 2024-02-18T01:27:12.189686+0000 mon.a (mon.0) 2031 : cluster [DBG] osdmap e306: 8 total, 8 up, 8 in 2024-02-18T01:27:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.257676+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.257787+0000 mon.a (mon.0) 2032 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.341237+0000 mon.a (mon.0) 2033 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.662033+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.663770+0000 mon.a (mon.0) 2035 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:12.672501+0000 mon.a (mon.0) 2036 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:13.213346+0000 mon.a (mon.0) 2037 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: cluster 2024-02-18T01:27:13.213826+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:13.214807+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:13.215374+0000 mon.a (mon.0) 2039 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[19734]: audit 2024-02-18T01:27:13.216003+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.257676+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.257787+0000 mon.a (mon.0) 2032 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.341237+0000 mon.a (mon.0) 2033 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.662033+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.663770+0000 mon.a (mon.0) 2035 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:12.672501+0000 mon.a (mon.0) 2036 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:13.213346+0000 mon.a (mon.0) 2037 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: cluster 2024-02-18T01:27:13.213826+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:13.214807+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:13.215374+0000 mon.a (mon.0) 2039 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:13 smithi105 bash[27669]: audit 2024-02-18T01:27:13.216003+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:13.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.257676+0000 mon.b (mon.1) 276 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.257787+0000 mon.a (mon.0) 2032 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:13.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.341237+0000 mon.a (mon.0) 2033 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:27:13.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.662033+0000 mon.a (mon.0) 2034 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:27:13.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.663770+0000 mon.a (mon.0) 2035 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:27:13.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:12.672501+0000 mon.a (mon.0) 2036 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:13.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:13.213346+0000 mon.a (mon.0) 2037 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:13.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: cluster 2024-02-18T01:27:13.213826+0000 mon.a (mon.0) 2038 : cluster [DBG] osdmap e307: 8 total, 8 up, 8 in 2024-02-18T01:27:13.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:13.214807+0000 mon.b (mon.1) 277 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:13.215374+0000 mon.a (mon.0) 2039 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:13.497 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:13 smithi155 bash[20805]: audit 2024-02-18T01:27:13.216003+0000 mon.a (mon.0) 2040 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: cluster 2024-02-18T01:27:13.740246+0000 mgr.y (mgr.24475) 410 : cluster [DBG] pgmap v538: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: audit 2024-02-18T01:27:14.213407+0000 mon.a (mon.0) 2041 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: audit 2024-02-18T01:27:14.213532+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: cluster 2024-02-18T01:27:14.213603+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: audit 2024-02-18T01:27:14.214263+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: audit 2024-02-18T01:27:14.216675+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:14.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:14 smithi155 bash[20805]: audit 2024-02-18T01:27:14.217017+0000 mon.a (mon.0) 2045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: cluster 2024-02-18T01:27:13.740246+0000 mgr.y (mgr.24475) 410 : cluster [DBG] pgmap v538: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: audit 2024-02-18T01:27:14.213407+0000 mon.a (mon.0) 2041 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: audit 2024-02-18T01:27:14.213532+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: cluster 2024-02-18T01:27:14.213603+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: audit 2024-02-18T01:27:14.214263+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: audit 2024-02-18T01:27:14.216675+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[19734]: audit 2024-02-18T01:27:14.217017+0000 mon.a (mon.0) 2045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: cluster 2024-02-18T01:27:13.740246+0000 mgr.y (mgr.24475) 410 : cluster [DBG] pgmap v538: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: audit 2024-02-18T01:27:14.213407+0000 mon.a (mon.0) 2041 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: audit 2024-02-18T01:27:14.213532+0000 mon.a (mon.0) 2042 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:14.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: cluster 2024-02-18T01:27:14.213603+0000 mon.a (mon.0) 2043 : cluster [DBG] osdmap e308: 8 total, 8 up, 8 in 2024-02-18T01:27:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: audit 2024-02-18T01:27:14.214263+0000 mon.a (mon.0) 2044 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]: dispatch 2024-02-18T01:27:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: audit 2024-02-18T01:27:14.216675+0000 mon.b (mon.1) 278 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:14.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:14 smithi105 bash[27669]: audit 2024-02-18T01:27:14.217017+0000 mon.a (mon.0) 2045 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]: dispatch 2024-02-18T01:27:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: cluster 2024-02-18T01:27:15.207505+0000 mon.a (mon.0) 2046 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: audit 2024-02-18T01:27:15.215296+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: audit 2024-02-18T01:27:15.215438+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]': finished 2024-02-18T01:27:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: cluster 2024-02-18T01:27:15.215507+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-18T01:27:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: audit 2024-02-18T01:27:15.218314+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[19734]: audit 2024-02-18T01:27:15.219364+0000 mon.a (mon.0) 2050 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: cluster 2024-02-18T01:27:15.207505+0000 mon.a (mon.0) 2046 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: audit 2024-02-18T01:27:15.215296+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: audit 2024-02-18T01:27:15.215438+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]': finished 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: cluster 2024-02-18T01:27:15.215507+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: audit 2024-02-18T01:27:15.218314+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:15 smithi105 bash[27669]: audit 2024-02-18T01:27:15.219364+0000 mon.a (mon.0) 2050 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: cluster 2024-02-18T01:27:15.207505+0000 mon.a (mon.0) 2046 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: audit 2024-02-18T01:27:15.215296+0000 mon.a (mon.0) 2047 : audit [INF] from='client.? v1:172.21.15.105:0/3603784567' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi105-61657-58"}]': finished 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: audit 2024-02-18T01:27:15.215438+0000 mon.a (mon.0) 2048 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-45", "mode": "writeback"}]': finished 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: cluster 2024-02-18T01:27:15.215507+0000 mon.a (mon.0) 2049 : cluster [DBG] osdmap e309: 8 total, 8 up, 8 in 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: audit 2024-02-18T01:27:15.218314+0000 mon.b (mon.1) 279 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:15 smithi155 bash[20805]: audit 2024-02-18T01:27:15.219364+0000 mon.a (mon.0) 2050 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:16] "GET /metrics HTTP/1.1" 200 187513 "" "Prometheus/2.33.4" 2024-02-18T01:27:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: cluster 2024-02-18T01:27:15.740936+0000 mgr.y (mgr.24475) 411 : cluster [DBG] pgmap v541: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:27:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: audit 2024-02-18T01:27:16.221272+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: cluster 2024-02-18T01:27:16.222199+0000 mon.a (mon.0) 2052 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: audit 2024-02-18T01:27:16.223062+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.105:0/4207640073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: audit 2024-02-18T01:27:16.224235+0000 mon.a (mon.0) 2053 : audit [INF] from='client.48203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: audit 2024-02-18T01:27:16.225929+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[27669]: audit 2024-02-18T01:27:16.228842+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: cluster 2024-02-18T01:27:15.740936+0000 mgr.y (mgr.24475) 411 : cluster [DBG] pgmap v541: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: audit 2024-02-18T01:27:16.221272+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: cluster 2024-02-18T01:27:16.222199+0000 mon.a (mon.0) 2052 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: audit 2024-02-18T01:27:16.223062+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.105:0/4207640073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: audit 2024-02-18T01:27:16.224235+0000 mon.a (mon.0) 2053 : audit [INF] from='client.48203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: audit 2024-02-18T01:27:16.225929+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:16.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:16 smithi105 bash[19734]: audit 2024-02-18T01:27:16.228842+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: cluster 2024-02-18T01:27:15.740936+0000 mgr.y (mgr.24475) 411 : cluster [DBG] pgmap v541: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: audit 2024-02-18T01:27:16.221272+0000 mon.a (mon.0) 2051 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: cluster 2024-02-18T01:27:16.222199+0000 mon.a (mon.0) 2052 : cluster [DBG] osdmap e310: 8 total, 8 up, 8 in 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: audit 2024-02-18T01:27:16.223062+0000 mon.c (mon.2) 368 : audit [INF] from='client.? v1:172.21.15.105:0/4207640073' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: audit 2024-02-18T01:27:16.224235+0000 mon.a (mon.0) 2053 : audit [INF] from='client.48203 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: audit 2024-02-18T01:27:16.225929+0000 mon.b (mon.1) 280 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:16.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:16 smithi155 bash[20805]: audit 2024-02-18T01:27:16.228842+0000 mon.a (mon.0) 2054 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:18.232 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: audit 2024-02-18T01:27:17.225216+0000 mon.a (mon.0) 2055 : audit [INF] from='client.48203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: audit 2024-02-18T01:27:17.225687+0000 mon.a (mon.0) 2056 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: cluster 2024-02-18T01:27:17.225872+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: audit 2024-02-18T01:27:17.229855+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: audit 2024-02-18T01:27:17.231691+0000 mon.a (mon.0) 2058 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: cluster 2024-02-18T01:27:17.743185+0000 mgr.y (mgr.24475) 412 : cluster [DBG] pgmap v544: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[19734]: audit 2024-02-18T01:27:17.788414+0000 mgr.y (mgr.24475) 413 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: audit 2024-02-18T01:27:17.225216+0000 mon.a (mon.0) 2055 : audit [INF] from='client.48203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: audit 2024-02-18T01:27:17.225687+0000 mon.a (mon.0) 2056 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: cluster 2024-02-18T01:27:17.225872+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: audit 2024-02-18T01:27:17.229855+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: audit 2024-02-18T01:27:17.231691+0000 mon.a (mon.0) 2058 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: cluster 2024-02-18T01:27:17.743185+0000 mgr.y (mgr.24475) 412 : cluster [DBG] pgmap v544: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:18.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:18 smithi105 bash[27669]: audit 2024-02-18T01:27:17.788414+0000 mgr.y (mgr.24475) 413 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: audit 2024-02-18T01:27:17.225216+0000 mon.a (mon.0) 2055 : audit [INF] from='client.48203 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi105-61657-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: audit 2024-02-18T01:27:17.225687+0000 mon.a (mon.0) 2056 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:18.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: cluster 2024-02-18T01:27:17.225872+0000 mon.a (mon.0) 2057 : cluster [DBG] osdmap e311: 8 total, 8 up, 8 in 2024-02-18T01:27:18.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: audit 2024-02-18T01:27:17.229855+0000 mon.b (mon.1) 281 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: audit 2024-02-18T01:27:17.231691+0000 mon.a (mon.0) 2058 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:18.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: cluster 2024-02-18T01:27:17.743185+0000 mgr.y (mgr.24475) 412 : cluster [DBG] pgmap v544: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:18.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:18 smithi155 bash[20805]: audit 2024-02-18T01:27:17.788414+0000 mgr.y (mgr.24475) 413 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: cluster 2024-02-18T01:27:18.216468+0000 mon.a (mon.0) 2059 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.224947+0000 mon.a (mon.0) 2060 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: cluster 2024-02-18T01:27:18.225156+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.228617+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.238715+0000 mon.a (mon.0) 2062 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.252845+0000 mon.c (mon.2) 369 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.253174+0000 mon.a (mon.0) 2063 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.254283+0000 mon.c (mon.2) 370 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.254576+0000 mon.a (mon.0) 2064 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.255025+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[19734]: audit 2024-02-18T01:27:18.255312+0000 mon.a (mon.0) 2065 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: cluster 2024-02-18T01:27:18.216468+0000 mon.a (mon.0) 2059 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.224947+0000 mon.a (mon.0) 2060 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: cluster 2024-02-18T01:27:18.225156+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.228617+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.238715+0000 mon.a (mon.0) 2062 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.252845+0000 mon.c (mon.2) 369 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.253174+0000 mon.a (mon.0) 2063 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.254283+0000 mon.c (mon.2) 370 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.254576+0000 mon.a (mon.0) 2064 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.255025+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:19.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:19 smithi105 bash[27669]: audit 2024-02-18T01:27:18.255312+0000 mon.a (mon.0) 2065 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: cluster 2024-02-18T01:27:18.216468+0000 mon.a (mon.0) 2059 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.224947+0000 mon.a (mon.0) 2060 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: cluster 2024-02-18T01:27:18.225156+0000 mon.a (mon.0) 2061 : cluster [DBG] osdmap e312: 8 total, 8 up, 8 in 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.228617+0000 mon.b (mon.1) 282 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.238715+0000 mon.a (mon.0) 2062 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.252845+0000 mon.c (mon.2) 369 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.253174+0000 mon.a (mon.0) 2063 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.254283+0000 mon.c (mon.2) 370 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.254576+0000 mon.a (mon.0) 2064 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:19.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.255025+0000 mon.c (mon.2) 371 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:19.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:19 smithi155 bash[20805]: audit 2024-02-18T01:27:18.255312+0000 mon.a (mon.0) 2065 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.239359+0000 mon.a (mon.0) 2066 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.239858+0000 mon.a (mon.0) 2067 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: cluster 2024-02-18T01:27:19.240022+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-18T01:27:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.241278+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.241441+0000 mon.c (mon.2) 372 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.242720+0000 mon.a (mon.0) 2069 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: audit 2024-02-18T01:27:19.243118+0000 mon.a (mon.0) 2070 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[27669]: cluster 2024-02-18T01:27:19.744078+0000 mgr.y (mgr.24475) 414 : cluster [DBG] pgmap v547: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.239359+0000 mon.a (mon.0) 2066 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.239858+0000 mon.a (mon.0) 2067 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: cluster 2024-02-18T01:27:19.240022+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.241278+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.241441+0000 mon.c (mon.2) 372 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.242720+0000 mon.a (mon.0) 2069 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: audit 2024-02-18T01:27:19.243118+0000 mon.a (mon.0) 2070 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:20 smithi105 bash[19734]: cluster 2024-02-18T01:27:19.744078+0000 mgr.y (mgr.24475) 414 : cluster [DBG] pgmap v547: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.239359+0000 mon.a (mon.0) 2066 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.239858+0000 mon.a (mon.0) 2067 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi105-61657-60", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: cluster 2024-02-18T01:27:19.240022+0000 mon.a (mon.0) 2068 : cluster [DBG] osdmap e313: 8 total, 8 up, 8 in 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.241278+0000 mon.b (mon.1) 283 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.241441+0000 mon.c (mon.2) 372 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.242720+0000 mon.a (mon.0) 2069 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: audit 2024-02-18T01:27:19.243118+0000 mon.a (mon.0) 2070 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:20 smithi155 bash[20805]: cluster 2024-02-18T01:27:19.744078+0000 mgr.y (mgr.24475) 414 : cluster [DBG] pgmap v547: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:20.237502+0000 mon.a (mon.0) 2071 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:27:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:20.241676+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: cluster 2024-02-18T01:27:20.659102+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-18T01:27:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:20.660735+0000 mon.a (mon.0) 2073 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:21.247491+0000 mon.a (mon.0) 2074 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:21.247782+0000 mon.a (mon.0) 2075 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: cluster 2024-02-18T01:27:21.248001+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:21.289800+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[19734]: audit 2024-02-18T01:27:21.290138+0000 mon.a (mon.0) 2077 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:20.237502+0000 mon.a (mon.0) 2071 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:20.241676+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: cluster 2024-02-18T01:27:20.659102+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:20.660735+0000 mon.a (mon.0) 2073 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:21.247491+0000 mon.a (mon.0) 2074 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:21.247782+0000 mon.a (mon.0) 2075 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:27:21.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: cluster 2024-02-18T01:27:21.248001+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-18T01:27:21.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:21.289800+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:21.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:21 smithi105 bash[27669]: audit 2024-02-18T01:27:21.290138+0000 mon.a (mon.0) 2077 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:20.237502+0000 mon.a (mon.0) 2071 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:20.241676+0000 mon.b (mon.1) 284 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: cluster 2024-02-18T01:27:20.659102+0000 mon.a (mon.0) 2072 : cluster [DBG] osdmap e314: 8 total, 8 up, 8 in 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:20.660735+0000 mon.a (mon.0) 2073 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:21.247491+0000 mon.a (mon.0) 2074 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi105-61657-60", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:21.247782+0000 mon.a (mon.0) 2075 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-45","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: cluster 2024-02-18T01:27:21.248001+0000 mon.a (mon.0) 2076 : cluster [DBG] osdmap e315: 8 total, 8 up, 8 in 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:21.289800+0000 mon.b (mon.1) 285 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:21.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:21 smithi155 bash[20805]: audit 2024-02-18T01:27:21.290138+0000 mon.a (mon.0) 2077 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[19734]: cluster 2024-02-18T01:27:21.744942+0000 mgr.y (mgr.24475) 415 : cluster [DBG] pgmap v550: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[19734]: audit 2024-02-18T01:27:22.246660+0000 mon.a (mon.0) 2078 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[19734]: cluster 2024-02-18T01:27:22.246872+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-18T01:27:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[19734]: audit 2024-02-18T01:27:22.247977+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[19734]: audit 2024-02-18T01:27:22.248249+0000 mon.a (mon.0) 2080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[27669]: cluster 2024-02-18T01:27:21.744942+0000 mgr.y (mgr.24475) 415 : cluster [DBG] pgmap v550: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[27669]: audit 2024-02-18T01:27:22.246660+0000 mon.a (mon.0) 2078 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[27669]: cluster 2024-02-18T01:27:22.246872+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[27669]: audit 2024-02-18T01:27:22.247977+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:22.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:22 smithi105 bash[27669]: audit 2024-02-18T01:27:22.248249+0000 mon.a (mon.0) 2080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:22 smithi155 bash[20805]: cluster 2024-02-18T01:27:21.744942+0000 mgr.y (mgr.24475) 415 : cluster [DBG] pgmap v550: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:22 smithi155 bash[20805]: audit 2024-02-18T01:27:22.246660+0000 mon.a (mon.0) 2078 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:22 smithi155 bash[20805]: cluster 2024-02-18T01:27:22.246872+0000 mon.a (mon.0) 2079 : cluster [DBG] osdmap e316: 8 total, 8 up, 8 in 2024-02-18T01:27:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:22 smithi155 bash[20805]: audit 2024-02-18T01:27:22.247977+0000 mon.b (mon.1) 286 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:22.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:22 smithi155 bash[20805]: audit 2024-02-18T01:27:22.248249+0000 mon.a (mon.0) 2080 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.249336+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: cluster 2024-02-18T01:27:23.249497+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.252402+0000 mon.c (mon.2) 373 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.252938+0000 mon.a (mon.0) 2083 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.281455+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.281719+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.282834+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: audit 2024-02-18T01:27:23.282889+0000 mon.a (mon.0) 2085 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[19734]: cluster 2024-02-18T01:27:23.745828+0000 mgr.y (mgr.24475) 416 : cluster [DBG] pgmap v553: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.249336+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: cluster 2024-02-18T01:27:23.249497+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.252402+0000 mon.c (mon.2) 373 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.252938+0000 mon.a (mon.0) 2083 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.281455+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.281719+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.282834+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: audit 2024-02-18T01:27:23.282889+0000 mon.a (mon.0) 2085 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:24 smithi105 bash[27669]: cluster 2024-02-18T01:27:23.745828+0000 mgr.y (mgr.24475) 416 : cluster [DBG] pgmap v553: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.249336+0000 mon.a (mon.0) 2081 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]': finished 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: cluster 2024-02-18T01:27:23.249497+0000 mon.a (mon.0) 2082 : cluster [DBG] osdmap e317: 8 total, 8 up, 8 in 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.252402+0000 mon.c (mon.2) 373 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.252938+0000 mon.a (mon.0) 2083 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.281455+0000 mon.b (mon.1) 287 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.281719+0000 mon.a (mon.0) 2084 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.282834+0000 mon.b (mon.1) 288 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: audit 2024-02-18T01:27:23.282889+0000 mon.a (mon.0) 2085 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-45"}]: dispatch 2024-02-18T01:27:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:24 smithi155 bash[20805]: cluster 2024-02-18T01:27:23.745828+0000 mgr.y (mgr.24475) 416 : cluster [DBG] pgmap v553: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:25.257 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190o.RoundTripAppendPP (3018 ms) 2024-02-18T01:27:25.257 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (2959 ms) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3010 ms) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3014 ms) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3427 ms) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 31 tests from LibRadosAio (128319 ms total) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2597 ms) 2024-02-18T01:27:25.258 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9029 ms) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14990 ms) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3020 ms) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29636 ms total) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3006 ms) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3006 ms total) 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-02-18T01:27:25.259 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (14005 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7052 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7073 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7042 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (2803 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7041 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7024 ms) 2024-02-18T01:27:25.260 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7010 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7047 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7066 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7019 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7070 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7058 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3008 ms) 2024-02-18T01:27:25.261 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-02-18T01:27:25.262 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7029 ms) 2024-02-18T01:27:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:24.253907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: cluster 2024-02-18T01:27:24.254145+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-18T01:27:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:24.255225+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:24.257652+0000 mon.a (mon.0) 2088 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:25.253639+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: cluster 2024-02-18T01:27:25.253787+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:25.258637+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[19734]: audit 2024-02-18T01:27:25.258982+0000 mon.a (mon.0) 2091 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:24.253907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: cluster 2024-02-18T01:27:24.254145+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:24.255225+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:24.257652+0000 mon.a (mon.0) 2088 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:25.253639+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: cluster 2024-02-18T01:27:25.253787+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-18T01:27:25.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:25.258637+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:25.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:25 smithi105 bash[27669]: audit 2024-02-18T01:27:25.258982+0000 mon.a (mon.0) 2091 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:24.253907+0000 mon.a (mon.0) 2086 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: cluster 2024-02-18T01:27:24.254145+0000 mon.a (mon.0) 2087 : cluster [DBG] osdmap e318: 8 total, 8 up, 8 in 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:24.255225+0000 mon.c (mon.2) 374 : audit [INF] from='client.? v1:172.21.15.105:0/1867687863' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:24.257652+0000 mon.a (mon.0) 2088 : audit [INF] from='client.48209 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]: dispatch 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:25.253639+0000 mon.a (mon.0) 2089 : audit [INF] from='client.48209 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi105-61657-60"}]': finished 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: cluster 2024-02-18T01:27:25.253787+0000 mon.a (mon.0) 2090 : cluster [DBG] osdmap e319: 8 total, 8 up, 8 in 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:25.258637+0000 mon.b (mon.1) 289 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:25.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:25 smithi155 bash[20805]: audit 2024-02-18T01:27:25.258982+0000 mon.a (mon.0) 2091 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.276952+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.277274+0000 mon.a (mon.0) 2092 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.278409+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.278729+0000 mon.a (mon.0) 2093 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.279187+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:25.279469+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: cluster 2024-02-18T01:27:25.746760+0000 mgr.y (mgr.24475) 417 : cluster [DBG] pgmap v556: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:26.258581+0000 mon.a (mon.0) 2095 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: audit 2024-02-18T01:27:26.258764+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[27669]: cluster 2024-02-18T01:27:26.258896+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.276952+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.277274+0000 mon.a (mon.0) 2092 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.278409+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.278729+0000 mon.a (mon.0) 2093 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.279187+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.680 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:25.279469+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: cluster 2024-02-18T01:27:25.746760+0000 mgr.y (mgr.24475) 417 : cluster [DBG] pgmap v556: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:26.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:26.258581+0000 mon.a (mon.0) 2095 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:26.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: audit 2024-02-18T01:27:26.258764+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:26.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[19734]: cluster 2024-02-18T01:27:26.258896+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.276952+0000 mon.c (mon.2) 375 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.277274+0000 mon.a (mon.0) 2092 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.278409+0000 mon.c (mon.2) 376 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.278729+0000 mon.a (mon.0) 2093 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.279187+0000 mon.c (mon.2) 377 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:25.279469+0000 mon.a (mon.0) 2094 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: cluster 2024-02-18T01:27:25.746760+0000 mgr.y (mgr.24475) 417 : cluster [DBG] pgmap v556: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 109 KiB/s rd, 0 B/s wr, 180 op/s 2024-02-18T01:27:26.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:26.258581+0000 mon.a (mon.0) 2095 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:26.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: audit 2024-02-18T01:27:26.258764+0000 mon.a (mon.0) 2096 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:26.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:26 smithi155 bash[20805]: cluster 2024-02-18T01:27:26.258896+0000 mon.a (mon.0) 2097 : cluster [DBG] osdmap e320: 8 total, 8 up, 8 in 2024-02-18T01:27:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:26] "GET /metrics HTTP/1.1" 200 190114 "" "Prometheus/2.33.4" 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[19734]: audit 2024-02-18T01:27:26.260583+0000 mon.c (mon.2) 378 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[19734]: audit 2024-02-18T01:27:26.261151+0000 mon.a (mon.0) 2098 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[19734]: audit 2024-02-18T01:27:26.280995+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[19734]: audit 2024-02-18T01:27:26.282224+0000 mon.a (mon.0) 2099 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[27669]: audit 2024-02-18T01:27:26.260583+0000 mon.c (mon.2) 378 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[27669]: audit 2024-02-18T01:27:26.261151+0000 mon.a (mon.0) 2098 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[27669]: audit 2024-02-18T01:27:26.280995+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:27.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:27 smithi105 bash[27669]: audit 2024-02-18T01:27:26.282224+0000 mon.a (mon.0) 2099 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:27.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:27 smithi155 bash[20805]: audit 2024-02-18T01:27:26.260583+0000 mon.c (mon.2) 378 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:27 smithi155 bash[20805]: audit 2024-02-18T01:27:26.261151+0000 mon.a (mon.0) 2098 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:27 smithi155 bash[20805]: audit 2024-02-18T01:27:26.280995+0000 mon.b (mon.1) 290 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:27.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:27 smithi155 bash[20805]: audit 2024-02-18T01:27:26.282224+0000 mon.a (mon.0) 2099 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:27.686222+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:28.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: cluster 2024-02-18T01:27:27.686460+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-18T01:27:28.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:27.687893+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:27.688615+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: cluster 2024-02-18T01:27:27.748919+0000 mgr.y (mgr.24475) 418 : cluster [DBG] pgmap v559: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:27.793842+0000 mgr.y (mgr.24475) 419 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:28.262953+0000 mon.a (mon.0) 2103 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:28.263086+0000 mon.a (mon.0) 2104 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: cluster 2024-02-18T01:27:28.263168+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-18T01:27:28.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:28.266911+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:28.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[27669]: audit 2024-02-18T01:27:28.267908+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:28.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:27.686222+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:28.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: cluster 2024-02-18T01:27:27.686460+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-18T01:27:28.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:27.687893+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:27.688615+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: cluster 2024-02-18T01:27:27.748919+0000 mgr.y (mgr.24475) 418 : cluster [DBG] pgmap v559: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:27.793842+0000 mgr.y (mgr.24475) 419 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:28.262953+0000 mon.a (mon.0) 2103 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:28.263086+0000 mon.a (mon.0) 2104 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:28.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: cluster 2024-02-18T01:27:28.263168+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-18T01:27:28.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:28.266911+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:28.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:28 smithi105 bash[19734]: audit 2024-02-18T01:27:28.267908+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:28.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:27.686222+0000 mon.a (mon.0) 2100 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:28.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: cluster 2024-02-18T01:27:27.686460+0000 mon.a (mon.0) 2101 : cluster [DBG] osdmap e321: 8 total, 8 up, 8 in 2024-02-18T01:27:28.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:27.687893+0000 mon.b (mon.1) 291 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:27.688615+0000 mon.a (mon.0) 2102 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:28.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: cluster 2024-02-18T01:27:27.748919+0000 mgr.y (mgr.24475) 418 : cluster [DBG] pgmap v559: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:27.793842+0000 mgr.y (mgr.24475) 419 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:28.262953+0000 mon.a (mon.0) 2103 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi105-61657-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:28.263086+0000 mon.a (mon.0) 2104 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: cluster 2024-02-18T01:27:28.263168+0000 mon.a (mon.0) 2105 : cluster [DBG] osdmap e322: 8 total, 8 up, 8 in 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:28.266911+0000 mon.b (mon.1) 292 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:28.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:28 smithi155 bash[20805]: audit 2024-02-18T01:27:28.267908+0000 mon.a (mon.0) 2106 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]: dispatch 2024-02-18T01:27:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[27669]: cluster 2024-02-18T01:27:29.258345+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[27669]: audit 2024-02-18T01:27:29.273693+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]': finished 2024-02-18T01:27:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[27669]: cluster 2024-02-18T01:27:29.274324+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-18T01:27:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[19734]: cluster 2024-02-18T01:27:29.258345+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[19734]: audit 2024-02-18T01:27:29.273693+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]': finished 2024-02-18T01:27:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:29 smithi105 bash[19734]: cluster 2024-02-18T01:27:29.274324+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-18T01:27:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:29 smithi155 bash[20805]: cluster 2024-02-18T01:27:29.258345+0000 mon.a (mon.0) 2107 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:29 smithi155 bash[20805]: audit 2024-02-18T01:27:29.273693+0000 mon.a (mon.0) 2108 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-47", "mode": "readproxy"}]': finished 2024-02-18T01:27:30.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:29 smithi155 bash[20805]: cluster 2024-02-18T01:27:29.274324+0000 mon.a (mon.0) 2109 : cluster [DBG] osdmap e323: 8 total, 8 up, 8 in 2024-02-18T01:27:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[19734]: cluster 2024-02-18T01:27:29.749903+0000 mgr.y (mgr.24475) 420 : cluster [DBG] pgmap v562: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[19734]: cluster 2024-02-18T01:27:30.298186+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-18T01:27:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[19734]: audit 2024-02-18T01:27:30.302052+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[19734]: audit 2024-02-18T01:27:30.303110+0000 mon.a (mon.0) 2111 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[27669]: cluster 2024-02-18T01:27:29.749903+0000 mgr.y (mgr.24475) 420 : cluster [DBG] pgmap v562: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[27669]: cluster 2024-02-18T01:27:30.298186+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-18T01:27:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[27669]: audit 2024-02-18T01:27:30.302052+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:31.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:30 smithi105 bash[27669]: audit 2024-02-18T01:27:30.303110+0000 mon.a (mon.0) 2111 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:30 smithi155 bash[20805]: cluster 2024-02-18T01:27:29.749903+0000 mgr.y (mgr.24475) 420 : cluster [DBG] pgmap v562: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:27:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:30 smithi155 bash[20805]: cluster 2024-02-18T01:27:30.298186+0000 mon.a (mon.0) 2110 : cluster [DBG] osdmap e324: 8 total, 8 up, 8 in 2024-02-18T01:27:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:30 smithi155 bash[20805]: audit 2024-02-18T01:27:30.302052+0000 mon.c (mon.2) 379 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:31.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:30 smithi155 bash[20805]: audit 2024-02-18T01:27:30.303110+0000 mon.a (mon.0) 2111 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[19734]: audit 2024-02-18T01:27:31.297826+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[19734]: cluster 2024-02-18T01:27:31.297978+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-18T01:27:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[19734]: audit 2024-02-18T01:27:31.298856+0000 mon.c (mon.2) 380 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[19734]: audit 2024-02-18T01:27:31.299299+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[19734]: cluster 2024-02-18T01:27:31.750848+0000 mgr.y (mgr.24475) 421 : cluster [DBG] pgmap v565: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[27669]: audit 2024-02-18T01:27:31.297826+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[27669]: cluster 2024-02-18T01:27:31.297978+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-18T01:27:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[27669]: audit 2024-02-18T01:27:31.298856+0000 mon.c (mon.2) 380 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[27669]: audit 2024-02-18T01:27:31.299299+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:32 smithi105 bash[27669]: cluster 2024-02-18T01:27:31.750848+0000 mgr.y (mgr.24475) 421 : cluster [DBG] pgmap v565: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:32 smithi155 bash[20805]: audit 2024-02-18T01:27:31.297826+0000 mon.a (mon.0) 2112 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:32.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:32 smithi155 bash[20805]: cluster 2024-02-18T01:27:31.297978+0000 mon.a (mon.0) 2113 : cluster [DBG] osdmap e325: 8 total, 8 up, 8 in 2024-02-18T01:27:32.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:32 smithi155 bash[20805]: audit 2024-02-18T01:27:31.298856+0000 mon.c (mon.2) 380 : audit [INF] from='client.? v1:172.21.15.105:0/1305107465' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:32 smithi155 bash[20805]: audit 2024-02-18T01:27:31.299299+0000 mon.a (mon.0) 2114 : audit [INF] from='client.48215 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]: dispatch 2024-02-18T01:27:32.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:32 smithi155 bash[20805]: cluster 2024-02-18T01:27:31.750848+0000 mgr.y (mgr.24475) 421 : cluster [DBG] pgmap v565: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[27669]: audit 2024-02-18T01:27:32.306852+0000 mon.a (mon.0) 2115 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[27669]: cluster 2024-02-18T01:27:32.306951+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-18T01:27:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[27669]: audit 2024-02-18T01:27:32.331750+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[27669]: audit 2024-02-18T01:27:32.333249+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[27669]: audit 2024-02-18T01:27:32.333668+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:33.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[19734]: audit 2024-02-18T01:27:32.306852+0000 mon.a (mon.0) 2115 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:33.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[19734]: cluster 2024-02-18T01:27:32.306951+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-18T01:27:33.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[19734]: audit 2024-02-18T01:27:32.331750+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[19734]: audit 2024-02-18T01:27:32.333249+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:33 smithi105 bash[19734]: audit 2024-02-18T01:27:32.333668+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:33.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:33 smithi155 bash[20805]: audit 2024-02-18T01:27:32.306852+0000 mon.a (mon.0) 2115 : audit [INF] from='client.48215 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi105-61657-61"}]': finished 2024-02-18T01:27:33.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:33 smithi155 bash[20805]: cluster 2024-02-18T01:27:32.306951+0000 mon.a (mon.0) 2116 : cluster [DBG] osdmap e326: 8 total, 8 up, 8 in 2024-02-18T01:27:33.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:33 smithi155 bash[20805]: audit 2024-02-18T01:27:32.331750+0000 mon.a (mon.0) 2117 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:33 smithi155 bash[20805]: audit 2024-02-18T01:27:32.333249+0000 mon.a (mon.0) 2118 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:33.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:33 smithi155 bash[20805]: audit 2024-02-18T01:27:32.333668+0000 mon.a (mon.0) 2119 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[27669]: audit 2024-02-18T01:27:33.318732+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[27669]: cluster 2024-02-18T01:27:33.318853+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-18T01:27:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[27669]: audit 2024-02-18T01:27:33.320533+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[27669]: cluster 2024-02-18T01:27:33.751720+0000 mgr.y (mgr.24475) 422 : cluster [DBG] pgmap v568: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[19734]: audit 2024-02-18T01:27:33.318732+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[19734]: cluster 2024-02-18T01:27:33.318853+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-18T01:27:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[19734]: audit 2024-02-18T01:27:33.320533+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:34 smithi105 bash[19734]: cluster 2024-02-18T01:27:33.751720+0000 mgr.y (mgr.24475) 422 : cluster [DBG] pgmap v568: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:34 smithi155 bash[20805]: audit 2024-02-18T01:27:33.318732+0000 mon.a (mon.0) 2120 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi105-61657-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:34 smithi155 bash[20805]: cluster 2024-02-18T01:27:33.318853+0000 mon.a (mon.0) 2121 : cluster [DBG] osdmap e327: 8 total, 8 up, 8 in 2024-02-18T01:27:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:34 smithi155 bash[20805]: audit 2024-02-18T01:27:33.320533+0000 mon.a (mon.0) 2122 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:34 smithi155 bash[20805]: cluster 2024-02-18T01:27:33.751720+0000 mgr.y (mgr.24475) 422 : cluster [DBG] pgmap v568: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[19734]: cluster 2024-02-18T01:27:34.746726+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-18T01:27:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[19734]: audit 2024-02-18T01:27:35.329962+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[19734]: cluster 2024-02-18T01:27:35.330222+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-18T01:27:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[27669]: cluster 2024-02-18T01:27:34.746726+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-18T01:27:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[27669]: audit 2024-02-18T01:27:35.329962+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:35 smithi105 bash[27669]: cluster 2024-02-18T01:27:35.330222+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-18T01:27:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:35 smithi155 bash[20805]: cluster 2024-02-18T01:27:34.746726+0000 mon.a (mon.0) 2123 : cluster [DBG] osdmap e328: 8 total, 8 up, 8 in 2024-02-18T01:27:36.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:35 smithi155 bash[20805]: audit 2024-02-18T01:27:35.329962+0000 mon.a (mon.0) 2124 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi105-61657-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:36.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:35 smithi155 bash[20805]: cluster 2024-02-18T01:27:35.330222+0000 mon.a (mon.0) 2125 : cluster [DBG] osdmap e329: 8 total, 8 up, 8 in 2024-02-18T01:27:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[27669]: cluster 2024-02-18T01:27:35.752549+0000 mgr.y (mgr.24475) 423 : cluster [DBG] pgmap v571: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[27669]: cluster 2024-02-18T01:27:36.333865+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-18T01:27:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[27669]: audit 2024-02-18T01:27:36.509756+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[19734]: cluster 2024-02-18T01:27:35.752549+0000 mgr.y (mgr.24475) 423 : cluster [DBG] pgmap v571: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[19734]: cluster 2024-02-18T01:27:36.333865+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-18T01:27:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[19734]: audit 2024-02-18T01:27:36.509756+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:36.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:36] "GET /metrics HTTP/1.1" 200 190379 "" "Prometheus/2.33.4" 2024-02-18T01:27:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:36 smithi155 bash[20805]: cluster 2024-02-18T01:27:35.752549+0000 mgr.y (mgr.24475) 423 : cluster [DBG] pgmap v571: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-02-18T01:27:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:36 smithi155 bash[20805]: cluster 2024-02-18T01:27:36.333865+0000 mon.a (mon.0) 2126 : cluster [DBG] osdmap e330: 8 total, 8 up, 8 in 2024-02-18T01:27:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:36 smithi155 bash[20805]: audit 2024-02-18T01:27:36.509756+0000 mon.a (mon.0) 2127 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:38.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[19734]: cluster 2024-02-18T01:27:37.337809+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-18T01:27:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[19734]: audit 2024-02-18T01:27:37.339287+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[19734]: cluster 2024-02-18T01:27:37.754802+0000 mgr.y (mgr.24475) 424 : cluster [DBG] pgmap v574: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:38.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[19734]: audit 2024-02-18T01:27:37.805155+0000 mgr.y (mgr.24475) 425 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:38.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[27669]: cluster 2024-02-18T01:27:37.337809+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-18T01:27:38.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[27669]: audit 2024-02-18T01:27:37.339287+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:38.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[27669]: cluster 2024-02-18T01:27:37.754802+0000 mgr.y (mgr.24475) 424 : cluster [DBG] pgmap v574: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:38.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:38 smithi105 bash[27669]: audit 2024-02-18T01:27:37.805155+0000 mgr.y (mgr.24475) 425 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:38.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:38 smithi155 bash[20805]: cluster 2024-02-18T01:27:37.337809+0000 mon.a (mon.0) 2128 : cluster [DBG] osdmap e331: 8 total, 8 up, 8 in 2024-02-18T01:27:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:38 smithi155 bash[20805]: audit 2024-02-18T01:27:37.339287+0000 mon.a (mon.0) 2129 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:38 smithi155 bash[20805]: cluster 2024-02-18T01:27:37.754802+0000 mgr.y (mgr.24475) 424 : cluster [DBG] pgmap v574: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:38.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:38 smithi155 bash[20805]: audit 2024-02-18T01:27:37.805155+0000 mgr.y (mgr.24475) 425 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[19734]: audit 2024-02-18T01:27:38.339392+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[19734]: cluster 2024-02-18T01:27:38.339480+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-18T01:27:39.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[19734]: audit 2024-02-18T01:27:38.341058+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:39.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[27669]: audit 2024-02-18T01:27:38.339392+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:39.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[27669]: cluster 2024-02-18T01:27:38.339480+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-18T01:27:39.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:39 smithi105 bash[27669]: audit 2024-02-18T01:27:38.341058+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:39.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:39 smithi155 bash[20805]: audit 2024-02-18T01:27:38.339392+0000 mon.a (mon.0) 2130 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:39.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:39 smithi155 bash[20805]: cluster 2024-02-18T01:27:38.339480+0000 mon.a (mon.0) 2131 : cluster [DBG] osdmap e332: 8 total, 8 up, 8 in 2024-02-18T01:27:39.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:39 smithi155 bash[20805]: audit 2024-02-18T01:27:38.341058+0000 mon.a (mon.0) 2132 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]: dispatch 2024-02-18T01:27:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.344058+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: cluster 2024-02-18T01:27:39.344483+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-18T01:27:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.364077+0000 mon.c (mon.2) 381 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.364468+0000 mon.a (mon.0) 2135 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.365807+0000 mon.c (mon.2) 382 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.366134+0000 mon.a (mon.0) 2136 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.366660+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.367009+0000 mon.a (mon.0) 2137 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.596050+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: audit 2024-02-18T01:27:39.596435+0000 mon.a (mon.0) 2138 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[19734]: cluster 2024-02-18T01:27:39.755631+0000 mgr.y (mgr.24475) 426 : cluster [DBG] pgmap v577: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.344058+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: cluster 2024-02-18T01:27:39.344483+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.364077+0000 mon.c (mon.2) 381 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.364468+0000 mon.a (mon.0) 2135 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.365807+0000 mon.c (mon.2) 382 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.366134+0000 mon.a (mon.0) 2136 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.366660+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.367009+0000 mon.a (mon.0) 2137 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.596050+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: audit 2024-02-18T01:27:39.596435+0000 mon.a (mon.0) 2138 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:40 smithi105 bash[27669]: cluster 2024-02-18T01:27:39.755631+0000 mgr.y (mgr.24475) 426 : cluster [DBG] pgmap v577: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.344058+0000 mon.a (mon.0) 2133 : audit [INF] from='client.? v1:172.21.15.105:0/1633477495' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi105-61657-62"}]': finished 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: cluster 2024-02-18T01:27:39.344483+0000 mon.a (mon.0) 2134 : cluster [DBG] osdmap e333: 8 total, 8 up, 8 in 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.364077+0000 mon.c (mon.2) 381 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.364468+0000 mon.a (mon.0) 2135 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.365807+0000 mon.c (mon.2) 382 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.366134+0000 mon.a (mon.0) 2136 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.366660+0000 mon.c (mon.2) 383 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.367009+0000 mon.a (mon.0) 2137 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:40.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.596050+0000 mon.b (mon.1) 293 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: audit 2024-02-18T01:27:39.596435+0000 mon.a (mon.0) 2138 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:40.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:40 smithi155 bash[20805]: cluster 2024-02-18T01:27:39.755631+0000 mgr.y (mgr.24475) 426 : cluster [DBG] pgmap v577: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.366183+0000 mon.a (mon.0) 2139 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.366295+0000 mon.a (mon.0) 2140 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: cluster 2024-02-18T01:27:40.366360+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-18T01:27:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.367997+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.368545+0000 mon.a (mon.0) 2142 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.369116+0000 mon.c (mon.2) 384 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:42.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[19734]: audit 2024-02-18T01:27:40.369696+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.366183+0000 mon.a (mon.0) 2139 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.366295+0000 mon.a (mon.0) 2140 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: cluster 2024-02-18T01:27:40.366360+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.367997+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.368545+0000 mon.a (mon.0) 2142 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.369116+0000 mon.c (mon.2) 384 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:42.243 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:41 smithi105 bash[27669]: audit 2024-02-18T01:27:40.369696+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:42.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:42.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.366183+0000 mon.a (mon.0) 2139 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi105-61657-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:42.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.366295+0000 mon.a (mon.0) 2140 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:27:42.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: cluster 2024-02-18T01:27:40.366360+0000 mon.a (mon.0) 2141 : cluster [DBG] osdmap e334: 8 total, 8 up, 8 in 2024-02-18T01:27:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.367997+0000 mon.b (mon.1) 294 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.368545+0000 mon.a (mon.0) 2142 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.369116+0000 mon.c (mon.2) 384 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:42.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:41 smithi155 bash[20805]: audit 2024-02-18T01:27:40.369696+0000 mon.a (mon.0) 2143 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:43.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: cluster 2024-02-18T01:27:41.362772+0000 mon.a (mon.0) 2144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:43.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:41.373365+0000 mon.a (mon.0) 2145 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:43.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:41.407124+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: cluster 2024-02-18T01:27:41.756376+0000 mgr.y (mgr.24475) 427 : cluster [DBG] pgmap v580: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: cluster 2024-02-18T01:27:41.794375+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:41.804447+0000 mon.a (mon.0) 2147 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:41.805902+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:41.806505+0000 mon.a (mon.0) 2148 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: audit 2024-02-18T01:27:42.377443+0000 mon.a (mon.0) 2149 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[19734]: cluster 2024-02-18T01:27:42.377845+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: cluster 2024-02-18T01:27:41.362772+0000 mon.a (mon.0) 2144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:41.373365+0000 mon.a (mon.0) 2145 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:41.407124+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: cluster 2024-02-18T01:27:41.756376+0000 mgr.y (mgr.24475) 427 : cluster [DBG] pgmap v580: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:43.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: cluster 2024-02-18T01:27:41.794375+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-18T01:27:43.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:41.804447+0000 mon.a (mon.0) 2147 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:41.805902+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:41.806505+0000 mon.a (mon.0) 2148 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: audit 2024-02-18T01:27:42.377443+0000 mon.a (mon.0) 2149 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:43.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:42 smithi105 bash[27669]: cluster 2024-02-18T01:27:42.377845+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: cluster 2024-02-18T01:27:41.362772+0000 mon.a (mon.0) 2144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:41.373365+0000 mon.a (mon.0) 2145 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]': finished 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:41.407124+0000 mon.b (mon.1) 295 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: cluster 2024-02-18T01:27:41.756376+0000 mgr.y (mgr.24475) 427 : cluster [DBG] pgmap v580: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: cluster 2024-02-18T01:27:41.794375+0000 mon.a (mon.0) 2146 : cluster [DBG] osdmap e335: 8 total, 8 up, 8 in 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:41.804447+0000 mon.a (mon.0) 2147 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:41.805902+0000 mon.b (mon.1) 296 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:41.806505+0000 mon.a (mon.0) 2148 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-47"}]: dispatch 2024-02-18T01:27:43.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: audit 2024-02-18T01:27:42.377443+0000 mon.a (mon.0) 2149 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi105-61657-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:43.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:42 smithi155 bash[20805]: cluster 2024-02-18T01:27:42.377845+0000 mon.a (mon.0) 2150 : cluster [DBG] osdmap e336: 8 total, 8 up, 8 in 2024-02-18T01:27:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[19734]: cluster 2024-02-18T01:27:43.381034+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-18T01:27:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[19734]: audit 2024-02-18T01:27:43.383664+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[19734]: audit 2024-02-18T01:27:43.384942+0000 mon.a (mon.0) 2152 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[19734]: cluster 2024-02-18T01:27:43.757276+0000 mgr.y (mgr.24475) 428 : cluster [DBG] pgmap v583: 236 pgs: 1 creating+peering, 39 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[27669]: cluster 2024-02-18T01:27:43.381034+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-18T01:27:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[27669]: audit 2024-02-18T01:27:43.383664+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[27669]: audit 2024-02-18T01:27:43.384942+0000 mon.a (mon.0) 2152 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:44 smithi105 bash[27669]: cluster 2024-02-18T01:27:43.757276+0000 mgr.y (mgr.24475) 428 : cluster [DBG] pgmap v583: 236 pgs: 1 creating+peering, 39 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:44.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:44 smithi155 bash[20805]: cluster 2024-02-18T01:27:43.381034+0000 mon.a (mon.0) 2151 : cluster [DBG] osdmap e337: 8 total, 8 up, 8 in 2024-02-18T01:27:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:44 smithi155 bash[20805]: audit 2024-02-18T01:27:43.383664+0000 mon.b (mon.1) 297 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:44 smithi155 bash[20805]: audit 2024-02-18T01:27:43.384942+0000 mon.a (mon.0) 2152 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:27:44.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:44 smithi155 bash[20805]: cluster 2024-02-18T01:27:43.757276+0000 mgr.y (mgr.24475) 428 : cluster [DBG] pgmap v583: 236 pgs: 1 creating+peering, 39 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:44.380499+0000 mon.a (mon.0) 2153 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: cluster 2024-02-18T01:27:44.380880+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-18T01:27:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:44.384254+0000 mon.c (mon.2) 385 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:44.386340+0000 mon.a (mon.0) 2155 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:44.414467+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:44.414523+0000 mon.a (mon.0) 2156 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:45.387069+0000 mon.a (mon.0) 2157 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: audit 2024-02-18T01:27:45.387180+0000 mon.a (mon.0) 2158 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[19734]: cluster 2024-02-18T01:27:45.387273+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:44.380499+0000 mon.a (mon.0) 2153 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: cluster 2024-02-18T01:27:44.380880+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:44.384254+0000 mon.c (mon.2) 385 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:44.386340+0000 mon.a (mon.0) 2155 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:44.414467+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:44.414523+0000 mon.a (mon.0) 2156 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:45.387069+0000 mon.a (mon.0) 2157 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: audit 2024-02-18T01:27:45.387180+0000 mon.a (mon.0) 2158 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:45.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:45 smithi105 bash[27669]: cluster 2024-02-18T01:27:45.387273+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:44.380499+0000 mon.a (mon.0) 2153 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: cluster 2024-02-18T01:27:44.380880+0000 mon.a (mon.0) 2154 : cluster [DBG] osdmap e338: 8 total, 8 up, 8 in 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:44.384254+0000 mon.c (mon.2) 385 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:44.386340+0000 mon.a (mon.0) 2155 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:44.414467+0000 mon.b (mon.1) 298 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:44.414523+0000 mon.a (mon.0) 2156 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:45.387069+0000 mon.a (mon.0) 2157 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: audit 2024-02-18T01:27:45.387180+0000 mon.a (mon.0) 2158 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:27:45.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:45 smithi155 bash[20805]: cluster 2024-02-18T01:27:45.387273+0000 mon.a (mon.0) 2159 : cluster [DBG] osdmap e339: 8 total, 8 up, 8 in 2024-02-18T01:27:46.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:45.388130+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:45.388528+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:45.388687+0000 mon.a (mon.0) 2160 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:45.389778+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: cluster 2024-02-18T01:27:45.758166+0000 mgr.y (mgr.24475) 429 : cluster [DBG] pgmap v586: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:46.387117+0000 mon.a (mon.0) 2162 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:46.387256+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: cluster 2024-02-18T01:27:46.387356+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:46.388732+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[27669]: audit 2024-02-18T01:27:46.389740+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:46.680 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:46] "GET /metrics HTTP/1.1" 200 190379 "" "Prometheus/2.33.4" 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:45.388130+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:45.388528+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:45.388687+0000 mon.a (mon.0) 2160 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:45.389778+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: cluster 2024-02-18T01:27:45.758166+0000 mgr.y (mgr.24475) 429 : cluster [DBG] pgmap v586: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:46.387117+0000 mon.a (mon.0) 2162 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:46.387256+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: cluster 2024-02-18T01:27:46.387356+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:46.388732+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:46.681 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:46 smithi105 bash[19734]: audit 2024-02-18T01:27:46.389740+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:45.388130+0000 mon.c (mon.2) 386 : audit [INF] from='client.? v1:172.21.15.105:0/4030165711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:45.388528+0000 mon.b (mon.1) 299 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:45.388687+0000 mon.a (mon.0) 2160 : audit [INF] from='client.48227 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]: dispatch 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:45.389778+0000 mon.a (mon.0) 2161 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: cluster 2024-02-18T01:27:45.758166+0000 mgr.y (mgr.24475) 429 : cluster [DBG] pgmap v586: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:46.387117+0000 mon.a (mon.0) 2162 : audit [INF] from='client.48227 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi105-61657-63"}]': finished 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:46.387256+0000 mon.a (mon.0) 2163 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:27:46.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: cluster 2024-02-18T01:27:46.387356+0000 mon.a (mon.0) 2164 : cluster [DBG] osdmap e340: 8 total, 8 up, 8 in 2024-02-18T01:27:46.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:46.388732+0000 mon.b (mon.1) 300 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:46.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:46 smithi155 bash[20805]: audit 2024-02-18T01:27:46.389740+0000 mon.a (mon.0) 2165 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]: dispatch 2024-02-18T01:27:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.408113+0000 mon.c (mon.2) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.408443+0000 mon.a (mon.0) 2166 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.409620+0000 mon.c (mon.2) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.409954+0000 mon.a (mon.0) 2167 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.410498+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:46.410814+0000 mon.a (mon.0) 2168 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: cluster 2024-02-18T01:27:47.381116+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:47.391227+0000 mon.a (mon.0) 2170 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]': finished 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:47.391485+0000 mon.a (mon.0) 2171 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: cluster 2024-02-18T01:27:47.391704+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:47.393673+0000 mon.c (mon.2) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[19734]: audit 2024-02-18T01:27:47.394146+0000 mon.a (mon.0) 2173 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.408113+0000 mon.c (mon.2) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.408443+0000 mon.a (mon.0) 2166 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.409620+0000 mon.c (mon.2) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.409954+0000 mon.a (mon.0) 2167 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.410498+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:46.410814+0000 mon.a (mon.0) 2168 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: cluster 2024-02-18T01:27:47.381116+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:47.391227+0000 mon.a (mon.0) 2170 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]': finished 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:47.391485+0000 mon.a (mon.0) 2171 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: cluster 2024-02-18T01:27:47.391704+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:47.393673+0000 mon.c (mon.2) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:47 smithi105 bash[27669]: audit 2024-02-18T01:27:47.394146+0000 mon.a (mon.0) 2173 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.408113+0000 mon.c (mon.2) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.408443+0000 mon.a (mon.0) 2166 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.409620+0000 mon.c (mon.2) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.409954+0000 mon.a (mon.0) 2167 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.410498+0000 mon.c (mon.2) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:46.410814+0000 mon.a (mon.0) 2168 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: cluster 2024-02-18T01:27:47.381116+0000 mon.a (mon.0) 2169 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:47.391227+0000 mon.a (mon.0) 2170 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-49", "mode": "writeback"}]': finished 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:47.391485+0000 mon.a (mon.0) 2171 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi105-61657-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: cluster 2024-02-18T01:27:47.391704+0000 mon.a (mon.0) 2172 : cluster [DBG] osdmap e341: 8 total, 8 up, 8 in 2024-02-18T01:27:47.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:47.393673+0000 mon.c (mon.2) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:47.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[20805]: audit 2024-02-18T01:27:47.394146+0000 mon.a (mon.0) 2173 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:48.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[27669]: audit 2024-02-18T01:27:47.442345+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[27669]: audit 2024-02-18T01:27:47.442532+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[27669]: cluster 2024-02-18T01:27:47.759144+0000 mgr.y (mgr.24475) 430 : cluster [DBG] pgmap v589: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[27669]: audit 2024-02-18T01:27:47.816501+0000 mgr.y (mgr.24475) 431 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[19734]: audit 2024-02-18T01:27:47.442345+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[19734]: audit 2024-02-18T01:27:47.442532+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[19734]: cluster 2024-02-18T01:27:47.759144+0000 mgr.y (mgr.24475) 430 : cluster [DBG] pgmap v589: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:49.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:48 smithi105 bash[19734]: audit 2024-02-18T01:27:47.816501+0000 mgr.y (mgr.24475) 431 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:48 smithi155 bash[20805]: audit 2024-02-18T01:27:47.442345+0000 mon.b (mon.1) 301 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:48 smithi155 bash[20805]: audit 2024-02-18T01:27:47.442532+0000 mon.a (mon.0) 2174 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:27:49.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:48 smithi155 bash[20805]: cluster 2024-02-18T01:27:47.759144+0000 mgr.y (mgr.24475) 430 : cluster [DBG] pgmap v589: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:49.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:48 smithi155 bash[20805]: audit 2024-02-18T01:27:47.816501+0000 mgr.y (mgr.24475) 431 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:48.824188+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: cluster 2024-02-18T01:27:48.824414+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-18T01:27:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:48.825975+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:48.826302+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:49.399549+0000 mon.a (mon.0) 2178 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:49.399772+0000 mon.a (mon.0) 2179 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: cluster 2024-02-18T01:27:49.399946+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:49.401768+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[19734]: audit 2024-02-18T01:27:49.403194+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:48.824188+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: cluster 2024-02-18T01:27:48.824414+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:48.825975+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:48.826302+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:49.399549+0000 mon.a (mon.0) 2178 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:50.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:49.399772+0000 mon.a (mon.0) 2179 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:50.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: cluster 2024-02-18T01:27:49.399946+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-18T01:27:50.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:49.401768+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:50.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:49 smithi105 bash[27669]: audit 2024-02-18T01:27:49.403194+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:48.824188+0000 mon.a (mon.0) 2175 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: cluster 2024-02-18T01:27:48.824414+0000 mon.a (mon.0) 2176 : cluster [DBG] osdmap e342: 8 total, 8 up, 8 in 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:48.825975+0000 mon.b (mon.1) 302 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:48.826302+0000 mon.a (mon.0) 2177 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:49.399549+0000 mon.a (mon.0) 2178 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi105-61657-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:49.399772+0000 mon.a (mon.0) 2179 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: cluster 2024-02-18T01:27:49.399946+0000 mon.a (mon.0) 2180 : cluster [DBG] osdmap e343: 8 total, 8 up, 8 in 2024-02-18T01:27:50.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:49.401768+0000 mon.b (mon.1) 303 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:50.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:49 smithi155 bash[20805]: audit 2024-02-18T01:27:49.403194+0000 mon.a (mon.0) 2181 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:27:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: cluster 2024-02-18T01:27:49.760008+0000 mgr.y (mgr.24475) 432 : cluster [DBG] pgmap v592: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: cluster 2024-02-18T01:27:50.390115+0000 mon.a (mon.0) 2182 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: audit 2024-02-18T01:27:50.404151+0000 mon.a (mon.0) 2183 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: cluster 2024-02-18T01:27:50.404669+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: audit 2024-02-18T01:27:50.405648+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[19734]: audit 2024-02-18T01:27:50.407455+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: cluster 2024-02-18T01:27:49.760008+0000 mgr.y (mgr.24475) 432 : cluster [DBG] pgmap v592: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: cluster 2024-02-18T01:27:50.390115+0000 mon.a (mon.0) 2182 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:51.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: audit 2024-02-18T01:27:50.404151+0000 mon.a (mon.0) 2183 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: cluster 2024-02-18T01:27:50.404669+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-18T01:27:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: audit 2024-02-18T01:27:50.405648+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:51.242 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:50 smithi105 bash[27669]: audit 2024-02-18T01:27:50.407455+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: cluster 2024-02-18T01:27:49.760008+0000 mgr.y (mgr.24475) 432 : cluster [DBG] pgmap v592: 236 pgs: 8 unknown, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:27:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: cluster 2024-02-18T01:27:50.390115+0000 mon.a (mon.0) 2182 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:27:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: audit 2024-02-18T01:27:50.404151+0000 mon.a (mon.0) 2183 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:27:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: cluster 2024-02-18T01:27:50.404669+0000 mon.a (mon.0) 2184 : cluster [DBG] osdmap e344: 8 total, 8 up, 8 in 2024-02-18T01:27:51.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: audit 2024-02-18T01:27:50.405648+0000 mon.b (mon.1) 304 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:51.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:50 smithi155 bash[20805]: audit 2024-02-18T01:27:50.407455+0000 mon.a (mon.0) 2185 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:27:52.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:27:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:27:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.400977+0000 mon.a (mon.0) 2186 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:52.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: cluster 2024-02-18T01:27:51.401256+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.402915+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.402936+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.405087+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.405971+0000 mon.a (mon.0) 2189 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: audit 2024-02-18T01:27:51.509790+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[19734]: cluster 2024-02-18T01:27:51.760851+0000 mgr.y (mgr.24475) 433 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.400977+0000 mon.a (mon.0) 2186 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: cluster 2024-02-18T01:27:51.401256+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.402915+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.402936+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.405087+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.405971+0000 mon.a (mon.0) 2189 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: audit 2024-02-18T01:27:51.509790+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:52.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:52 smithi105 bash[27669]: cluster 2024-02-18T01:27:51.760851+0000 mgr.y (mgr.24475) 433 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.400977+0000 mon.a (mon.0) 2186 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: cluster 2024-02-18T01:27:51.401256+0000 mon.a (mon.0) 2187 : cluster [DBG] osdmap e345: 8 total, 8 up, 8 in 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.402915+0000 mon.b (mon.1) 305 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.402936+0000 mon.c (mon.2) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.405087+0000 mon.a (mon.0) 2188 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.405971+0000 mon.a (mon.0) 2189 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: audit 2024-02-18T01:27:51.509790+0000 mon.a (mon.0) 2190 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:27:52.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:52 smithi155 bash[20805]: cluster 2024-02-18T01:27:51.760851+0000 mgr.y (mgr.24475) 433 : cluster [DBG] pgmap v595: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: audit 2024-02-18T01:27:52.403927+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: audit 2024-02-18T01:27:52.404050+0000 mon.a (mon.0) 2192 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: cluster 2024-02-18T01:27:52.404138+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: audit 2024-02-18T01:27:52.404938+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: audit 2024-02-18T01:27:52.406613+0000 mon.a (mon.0) 2194 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: audit 2024-02-18T01:27:53.407968+0000 mon.a (mon.0) 2195 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[19734]: cluster 2024-02-18T01:27:53.408220+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: audit 2024-02-18T01:27:52.403927+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: audit 2024-02-18T01:27:52.404050+0000 mon.a (mon.0) 2192 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: cluster 2024-02-18T01:27:52.404138+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: audit 2024-02-18T01:27:52.404938+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: audit 2024-02-18T01:27:52.406613+0000 mon.a (mon.0) 2194 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: audit 2024-02-18T01:27:53.407968+0000 mon.a (mon.0) 2195 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:53 smithi105 bash[27669]: cluster 2024-02-18T01:27:53.408220+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: audit 2024-02-18T01:27:52.403927+0000 mon.a (mon.0) 2191 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-49","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: audit 2024-02-18T01:27:52.404050+0000 mon.a (mon.0) 2192 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: cluster 2024-02-18T01:27:52.404138+0000 mon.a (mon.0) 2193 : cluster [DBG] osdmap e346: 8 total, 8 up, 8 in 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: audit 2024-02-18T01:27:52.404938+0000 mon.c (mon.2) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2557765049' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: audit 2024-02-18T01:27:52.406613+0000 mon.a (mon.0) 2194 : audit [INF] from='client.48233 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]: dispatch 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: audit 2024-02-18T01:27:53.407968+0000 mon.a (mon.0) 2195 : audit [INF] from='client.48233 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi105-61657-64"}]': finished 2024-02-18T01:27:53.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:53 smithi155 bash[20805]: cluster 2024-02-18T01:27:53.408220+0000 mon.a (mon.0) 2196 : cluster [DBG] osdmap e347: 8 total, 8 up, 8 in 2024-02-18T01:27:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.428538+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.428923+0000 mon.a (mon.0) 2197 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.430099+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.430419+0000 mon.a (mon.0) 2198 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.430963+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: audit 2024-02-18T01:27:53.431286+0000 mon.a (mon.0) 2199 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[19734]: cluster 2024-02-18T01:27:53.761770+0000 mgr.y (mgr.24475) 434 : cluster [DBG] pgmap v598: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.428538+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.428923+0000 mon.a (mon.0) 2197 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.430099+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.430419+0000 mon.a (mon.0) 2198 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.430963+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: audit 2024-02-18T01:27:53.431286+0000 mon.a (mon.0) 2199 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:54 smithi105 bash[27669]: cluster 2024-02-18T01:27:53.761770+0000 mgr.y (mgr.24475) 434 : cluster [DBG] pgmap v598: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.428538+0000 mon.c (mon.2) 393 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.428923+0000 mon.a (mon.0) 2197 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.430099+0000 mon.c (mon.2) 394 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.430419+0000 mon.a (mon.0) 2198 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.430963+0000 mon.c (mon.2) 395 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: audit 2024-02-18T01:27:53.431286+0000 mon.a (mon.0) 2199 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:27:54.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:54 smithi155 bash[20805]: cluster 2024-02-18T01:27:53.761770+0000 mgr.y (mgr.24475) 434 : cluster [DBG] pgmap v598: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[27669]: cluster 2024-02-18T01:27:54.413985+0000 mon.a (mon.0) 2200 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:27:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[27669]: audit 2024-02-18T01:27:54.430186+0000 mon.a (mon.0) 2201 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:55.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[27669]: cluster 2024-02-18T01:27:54.430391+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[27669]: audit 2024-02-18T01:27:54.434085+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[27669]: audit 2024-02-18T01:27:54.434802+0000 mon.a (mon.0) 2203 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[19734]: cluster 2024-02-18T01:27:54.413985+0000 mon.a (mon.0) 2200 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[19734]: audit 2024-02-18T01:27:54.430186+0000 mon.a (mon.0) 2201 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[19734]: cluster 2024-02-18T01:27:54.430391+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[19734]: audit 2024-02-18T01:27:54.434085+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:55.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:55 smithi105 bash[19734]: audit 2024-02-18T01:27:54.434802+0000 mon.a (mon.0) 2203 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:55.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:55 smithi155 bash[20805]: cluster 2024-02-18T01:27:54.413985+0000 mon.a (mon.0) 2200 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:27:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:55 smithi155 bash[20805]: audit 2024-02-18T01:27:54.430186+0000 mon.a (mon.0) 2201 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi105-61657-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:27:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:55 smithi155 bash[20805]: cluster 2024-02-18T01:27:54.430391+0000 mon.a (mon.0) 2202 : cluster [DBG] osdmap e348: 8 total, 8 up, 8 in 2024-02-18T01:27:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:55 smithi155 bash[20805]: audit 2024-02-18T01:27:54.434085+0000 mon.c (mon.2) 396 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:55.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:55 smithi155 bash[20805]: audit 2024-02-18T01:27:54.434802+0000 mon.a (mon.0) 2203 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:56.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:56 smithi105 bash[19734]: cluster 2024-02-18T01:27:55.762585+0000 mgr.y (mgr.24475) 435 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:56.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:56 smithi105 bash[19734]: cluster 2024-02-18T01:27:55.871162+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-18T01:27:56.740 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:27:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:27:56] "GET /metrics HTTP/1.1" 200 188085 "" "Prometheus/2.33.4" 2024-02-18T01:27:56.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:56 smithi105 bash[27669]: cluster 2024-02-18T01:27:55.762585+0000 mgr.y (mgr.24475) 435 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:56.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:56 smithi105 bash[27669]: cluster 2024-02-18T01:27:55.871162+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-18T01:27:56.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:56 smithi155 bash[20805]: cluster 2024-02-18T01:27:55.762585+0000 mgr.y (mgr.24475) 435 : cluster [DBG] pgmap v601: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:27:56.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:56 smithi155 bash[20805]: cluster 2024-02-18T01:27:55.871162+0000 mon.a (mon.0) 2204 : cluster [DBG] osdmap e349: 8 total, 8 up, 8 in 2024-02-18T01:27:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:57 smithi105 bash[19734]: audit 2024-02-18T01:27:56.448808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:27:57.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:57 smithi105 bash[19734]: cluster 2024-02-18T01:27:56.448950+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-18T01:27:57.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:57 smithi105 bash[27669]: audit 2024-02-18T01:27:56.448808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:27:57.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:57 smithi105 bash[27669]: cluster 2024-02-18T01:27:56.448950+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-18T01:27:57.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:57 smithi155 bash[20805]: audit 2024-02-18T01:27:56.448808+0000 mon.a (mon.0) 2205 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi105-61657-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:27:57.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:57 smithi155 bash[20805]: cluster 2024-02-18T01:27:56.448950+0000 mon.a (mon.0) 2206 : cluster [DBG] osdmap e350: 8 total, 8 up, 8 in 2024-02-18T01:27:58.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:27:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[19734]: cluster 2024-02-18T01:27:57.454302+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[19734]: cluster 2024-02-18T01:27:57.763465+0000 mgr.y (mgr.24475) 436 : cluster [DBG] pgmap v604: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[19734]: audit 2024-02-18T01:27:57.822320+0000 mgr.y (mgr.24475) 437 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[27669]: cluster 2024-02-18T01:27:57.454302+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[27669]: cluster 2024-02-18T01:27:57.763465+0000 mgr.y (mgr.24475) 436 : cluster [DBG] pgmap v604: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:27:58.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:58 smithi105 bash[27669]: audit 2024-02-18T01:27:57.822320+0000 mgr.y (mgr.24475) 437 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:58 smithi155 bash[20805]: cluster 2024-02-18T01:27:57.454302+0000 mon.a (mon.0) 2207 : cluster [DBG] osdmap e351: 8 total, 8 up, 8 in 2024-02-18T01:27:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:58 smithi155 bash[20805]: cluster 2024-02-18T01:27:57.763465+0000 mgr.y (mgr.24475) 436 : cluster [DBG] pgmap v604: 236 pgs: 8 creating+peering, 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:27:58.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:58 smithi155 bash[20805]: audit 2024-02-18T01:27:57.822320+0000 mgr.y (mgr.24475) 437 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:27:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[27669]: cluster 2024-02-18T01:27:58.466738+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-18T01:27:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[27669]: audit 2024-02-18T01:27:58.471631+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:59.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[27669]: audit 2024-02-18T01:27:58.477485+0000 mon.a (mon.0) 2209 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:59.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[19734]: cluster 2024-02-18T01:27:58.466738+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-18T01:27:59.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[19734]: audit 2024-02-18T01:27:58.471631+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:59.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:27:59 smithi105 bash[19734]: audit 2024-02-18T01:27:58.477485+0000 mon.a (mon.0) 2209 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:59.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:59 smithi155 bash[20805]: cluster 2024-02-18T01:27:58.466738+0000 mon.a (mon.0) 2208 : cluster [DBG] osdmap e352: 8 total, 8 up, 8 in 2024-02-18T01:27:59.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:59 smithi155 bash[20805]: audit 2024-02-18T01:27:58.471631+0000 mon.c (mon.2) 397 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:27:59.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:27:59 smithi155 bash[20805]: audit 2024-02-18T01:27:58.477485+0000 mon.a (mon.0) 2209 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7057 ms) 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7035 ms) 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-02-18T01:28:00.510 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7042 ms) 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7021 ms) 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7096 ms) 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139598 ms total) 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [==========] 56 tests from 4 test suites ran. (300559 ms total) 2024-02-18T01:28:00.511 INFO:tasks.workunit.client.0.smithi105.stdout: api_aio_pp: [ PASSED ] 56 tests. 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61668 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61668 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61845 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61845 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61765 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61765 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61692 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61692 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61985 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61985 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61660 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61660 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62088 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62088 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61933 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61933 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.514 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62166 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62166 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62045 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62045 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62321 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62321 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61644 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61644 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62245 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62245 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.515 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=62125 2024-02-18T01:28:00.516 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 62125 2024-02-18T01:28:00.516 INFO:tasks.workunit.client.0.smithi105.stderr:+ for t in "${!pids[@]}" 2024-02-18T01:28:00.516 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=61805 2024-02-18T01:28:00.516 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 61805 2024-02-18T01:28:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[19734]: audit 2024-02-18T01:27:59.492230+0000 mon.a (mon.0) 2210 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[19734]: cluster 2024-02-18T01:27:59.492391+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-18T01:28:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[19734]: audit 2024-02-18T01:27:59.493479+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[19734]: audit 2024-02-18T01:27:59.494162+0000 mon.a (mon.0) 2212 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[19734]: cluster 2024-02-18T01:27:59.764369+0000 mgr.y (mgr.24475) 438 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:28:00.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[27669]: audit 2024-02-18T01:27:59.492230+0000 mon.a (mon.0) 2210 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:00.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[27669]: cluster 2024-02-18T01:27:59.492391+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-18T01:28:00.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[27669]: audit 2024-02-18T01:27:59.493479+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[27669]: audit 2024-02-18T01:27:59.494162+0000 mon.a (mon.0) 2212 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:00 smithi105 bash[27669]: cluster 2024-02-18T01:27:59.764369+0000 mgr.y (mgr.24475) 438 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:28:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:00 smithi155 bash[20805]: audit 2024-02-18T01:27:59.492230+0000 mon.a (mon.0) 2210 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:00 smithi155 bash[20805]: cluster 2024-02-18T01:27:59.492391+0000 mon.a (mon.0) 2211 : cluster [DBG] osdmap e353: 8 total, 8 up, 8 in 2024-02-18T01:28:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:00 smithi155 bash[20805]: audit 2024-02-18T01:27:59.493479+0000 mon.c (mon.2) 398 : audit [INF] from='client.? v1:172.21.15.105:0/1894165480' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:00 smithi155 bash[20805]: audit 2024-02-18T01:27:59.494162+0000 mon.a (mon.0) 2212 : audit [INF] from='client.48239 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]: dispatch 2024-02-18T01:28:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:00 smithi155 bash[20805]: cluster 2024-02-18T01:27:59.764369+0000 mgr.y (mgr.24475) 438 : cluster [DBG] pgmap v607: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:28:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:01 smithi155 bash[20805]: audit 2024-02-18T01:28:00.497781+0000 mon.a (mon.0) 2213 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:01.878 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:01 smithi155 bash[20805]: cluster 2024-02-18T01:28:00.497897+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-18T01:28:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:01 smithi105 bash[19734]: audit 2024-02-18T01:28:00.497781+0000 mon.a (mon.0) 2213 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:01 smithi105 bash[19734]: cluster 2024-02-18T01:28:00.497897+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-18T01:28:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:01 smithi105 bash[27669]: audit 2024-02-18T01:28:00.497781+0000 mon.a (mon.0) 2213 : audit [INF] from='client.48239 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi105-61657-65"}]': finished 2024-02-18T01:28:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:01 smithi105 bash[27669]: cluster 2024-02-18T01:28:00.497897+0000 mon.a (mon.0) 2214 : cluster [DBG] osdmap e354: 8 total, 8 up, 8 in 2024-02-18T01:28:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[27669]: cluster 2024-02-18T01:28:01.765287+0000 mgr.y (mgr.24475) 439 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[27669]: audit 2024-02-18T01:28:02.434549+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[27669]: audit 2024-02-18T01:28:02.434971+0000 mon.a (mon.0) 2215 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:02.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[19734]: cluster 2024-02-18T01:28:01.765287+0000 mgr.y (mgr.24475) 439 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:02.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[19734]: audit 2024-02-18T01:28:02.434549+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:02.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:02 smithi105 bash[19734]: audit 2024-02-18T01:28:02.434971+0000 mon.a (mon.0) 2215 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:02 smithi155 bash[20805]: cluster 2024-02-18T01:28:01.765287+0000 mgr.y (mgr.24475) 439 : cluster [DBG] pgmap v609: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:02.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:02 smithi155 bash[20805]: audit 2024-02-18T01:28:02.434549+0000 mon.b (mon.1) 306 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:02.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:02 smithi155 bash[20805]: audit 2024-02-18T01:28:02.434971+0000 mon.a (mon.0) 2215 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[19734]: audit 2024-02-18T01:28:02.524046+0000 mon.a (mon.0) 2216 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:28:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[19734]: cluster 2024-02-18T01:28:02.524265+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-18T01:28:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[19734]: audit 2024-02-18T01:28:02.525641+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:03.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[19734]: audit 2024-02-18T01:28:02.526366+0000 mon.a (mon.0) 2218 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[27669]: audit 2024-02-18T01:28:02.524046+0000 mon.a (mon.0) 2216 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:28:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[27669]: cluster 2024-02-18T01:28:02.524265+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-18T01:28:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[27669]: audit 2024-02-18T01:28:02.525641+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:03.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:03 smithi105 bash[27669]: audit 2024-02-18T01:28:02.526366+0000 mon.a (mon.0) 2218 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:03.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:03 smithi155 bash[20805]: audit 2024-02-18T01:28:02.524046+0000 mon.a (mon.0) 2216 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:28:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:03 smithi155 bash[20805]: cluster 2024-02-18T01:28:02.524265+0000 mon.a (mon.0) 2217 : cluster [DBG] osdmap e355: 8 total, 8 up, 8 in 2024-02-18T01:28:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:03 smithi155 bash[20805]: audit 2024-02-18T01:28:02.525641+0000 mon.b (mon.1) 307 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:03.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:03 smithi155 bash[20805]: audit 2024-02-18T01:28:02.526366+0000 mon.a (mon.0) 2218 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:04.541 INFO:tasks.workunit.client.0.smithi105.stdout::b4c720e9:::166:head 2024-02-18T01:28:04.541 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e694b040:::167:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:afa38db2:::168:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:77ba9f53:::169:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:87495034:::170:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7c96bf0e:::171:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:dbe346cc:::172:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:e943ec24:::173:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f97a9c0c:::174:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:6f26e74d:::175:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4f95e106:::176:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:0e6f2f8f:::177:head 2024-02-18T01:28:04.542 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:05db05f1:::178:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:38a78d66:::179:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:d095610b:::180:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:a1a9d709:::181:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:1e5d39db:::182:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:f7df4fb9:::183:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:03a7f161:::184:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ba70721e:::185:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:28e5662d:::186:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:973d52de:::187:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:4303eb1c:::188:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:b990b48e:::189:head 2024-02-18T01:28:04.543 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:29b8165b:::190:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:3547f197:::191:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:7e260936:::192:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:1abec7b1:::193:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:10fdda93:::194:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:15817eea:::195:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:770bab57:::196:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:ed9e13e7:::197:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:71471a8f:::198:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: checking for 190:10fb1d02:::199:head 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8483 ms) 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-02-18T01:28:04.544 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: first is 1708219617 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,1708219625,1708219626,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,1708219625,1708219626,0 2024-02-18T01:28:04.545 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219617,1708219619,1708219620,1708219622,1708219623,1708219625,1708219626,0 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708219620,1708219622,1708219623,1708219625,1708219626,1708219628,1708219629,0 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: first now 1708219620, trimmed 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21140 ms) 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: foo0 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (13653 ms) 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ProxyRead (18124 ms) 2024-02-18T01:28:04.546 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-02-18T01:28:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: audit 2024-02-18T01:28:03.526361+0000 mon.a (mon.0) 2219 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:28:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: cluster 2024-02-18T01:28:03.526564+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-18T01:28:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: audit 2024-02-18T01:28:03.559134+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: audit 2024-02-18T01:28:03.559306+0000 mon.a (mon.0) 2221 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: audit 2024-02-18T01:28:03.560402+0000 mon.a (mon.0) 2222 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: audit 2024-02-18T01:28:03.560414+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[19734]: cluster 2024-02-18T01:28:03.766080+0000 mgr.y (mgr.24475) 440 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: audit 2024-02-18T01:28:03.526361+0000 mon.a (mon.0) 2219 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: cluster 2024-02-18T01:28:03.526564+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: audit 2024-02-18T01:28:03.559134+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: audit 2024-02-18T01:28:03.559306+0000 mon.a (mon.0) 2221 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: audit 2024-02-18T01:28:03.560402+0000 mon.a (mon.0) 2222 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: audit 2024-02-18T01:28:03.560414+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:04 smithi105 bash[27669]: cluster 2024-02-18T01:28:03.766080+0000 mgr.y (mgr.24475) 440 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:28:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: audit 2024-02-18T01:28:03.526361+0000 mon.a (mon.0) 2219 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]': finished 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: cluster 2024-02-18T01:28:03.526564+0000 mon.a (mon.0) 2220 : cluster [DBG] osdmap e356: 8 total, 8 up, 8 in 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: audit 2024-02-18T01:28:03.559134+0000 mon.b (mon.1) 308 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: audit 2024-02-18T01:28:03.559306+0000 mon.a (mon.0) 2221 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: audit 2024-02-18T01:28:03.560402+0000 mon.a (mon.0) 2222 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: audit 2024-02-18T01:28:03.560414+0000 mon.b (mon.1) 309 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-49"}]: dispatch 2024-02-18T01:28:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:04 smithi155 bash[20805]: cluster 2024-02-18T01:28:03.766080+0000 mgr.y (mgr.24475) 440 : cluster [DBG] pgmap v612: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:28:06.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[19734]: cluster 2024-02-18T01:28:04.530722+0000 mon.a (mon.0) 2223 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:28:06.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[19734]: cluster 2024-02-18T01:28:04.539107+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-18T01:28:06.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[19734]: cluster 2024-02-18T01:28:05.541247+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-18T01:28:06.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[19734]: audit 2024-02-18T01:28:05.546232+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[19734]: audit 2024-02-18T01:28:05.547883+0000 mon.a (mon.0) 2226 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[27669]: cluster 2024-02-18T01:28:04.530722+0000 mon.a (mon.0) 2223 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:28:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[27669]: cluster 2024-02-18T01:28:04.539107+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-18T01:28:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[27669]: cluster 2024-02-18T01:28:05.541247+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-18T01:28:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[27669]: audit 2024-02-18T01:28:05.546232+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:05 smithi105 bash[27669]: audit 2024-02-18T01:28:05.547883+0000 mon.a (mon.0) 2226 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:05 smithi155 bash[20805]: cluster 2024-02-18T01:28:04.530722+0000 mon.a (mon.0) 2223 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:28:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:05 smithi155 bash[20805]: cluster 2024-02-18T01:28:04.539107+0000 mon.a (mon.0) 2224 : cluster [DBG] osdmap e357: 8 total, 8 up, 8 in 2024-02-18T01:28:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:05 smithi155 bash[20805]: cluster 2024-02-18T01:28:05.541247+0000 mon.a (mon.0) 2225 : cluster [DBG] osdmap e358: 8 total, 8 up, 8 in 2024-02-18T01:28:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:05 smithi155 bash[20805]: audit 2024-02-18T01:28:05.546232+0000 mon.b (mon.1) 310 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:05 smithi155 bash[20805]: audit 2024-02-18T01:28:05.547883+0000 mon.a (mon.0) 2226 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:06.984 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:06] "GET /metrics HTTP/1.1" 200 188207 "" "Prometheus/2.33.4" 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: cluster 2024-02-18T01:28:05.766896+0000 mgr.y (mgr.24475) 441 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: audit 2024-02-18T01:28:06.546061+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: cluster 2024-02-18T01:28:06.546377+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: audit 2024-02-18T01:28:06.587008+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: audit 2024-02-18T01:28:06.587061+0000 mon.a (mon.0) 2229 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: audit 2024-02-18T01:28:06.588024+0000 mon.a (mon.0) 2230 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[19734]: audit 2024-02-18T01:28:06.588087+0000 mon.b (mon.1) 312 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:06.985 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: cluster 2024-02-18T01:28:05.766896+0000 mgr.y (mgr.24475) 441 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:06.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: audit 2024-02-18T01:28:06.546061+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:06.986 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: cluster 2024-02-18T01:28:06.546377+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-18T01:28:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: audit 2024-02-18T01:28:06.587008+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: audit 2024-02-18T01:28:06.587061+0000 mon.a (mon.0) 2229 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: audit 2024-02-18T01:28:06.588024+0000 mon.a (mon.0) 2230 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:06 smithi105 bash[27669]: audit 2024-02-18T01:28:06.588087+0000 mon.b (mon.1) 312 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: cluster 2024-02-18T01:28:05.766896+0000 mgr.y (mgr.24475) 441 : cluster [DBG] pgmap v615: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: audit 2024-02-18T01:28:06.546061+0000 mon.a (mon.0) 2227 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: cluster 2024-02-18T01:28:06.546377+0000 mon.a (mon.0) 2228 : cluster [DBG] osdmap e359: 8 total, 8 up, 8 in 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: audit 2024-02-18T01:28:06.587008+0000 mon.b (mon.1) 311 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: audit 2024-02-18T01:28:06.587061+0000 mon.a (mon.0) 2229 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: audit 2024-02-18T01:28:06.588024+0000 mon.a (mon.0) 2230 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:07.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:06 smithi155 bash[20805]: audit 2024-02-18T01:28:06.588087+0000 mon.b (mon.1) 312 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-51"}]: dispatch 2024-02-18T01:28:08.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[27669]: cluster 2024-02-18T01:28:07.558629+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-18T01:28:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[27669]: cluster 2024-02-18T01:28:07.767657+0000 mgr.y (mgr.24475) 442 : cluster [DBG] pgmap v618: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[27669]: audit 2024-02-18T01:28:07.831769+0000 mgr.y (mgr.24475) 443 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[19734]: cluster 2024-02-18T01:28:07.558629+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-18T01:28:08.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[19734]: cluster 2024-02-18T01:28:07.767657+0000 mgr.y (mgr.24475) 442 : cluster [DBG] pgmap v618: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:08.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:08 smithi105 bash[19734]: audit 2024-02-18T01:28:07.831769+0000 mgr.y (mgr.24475) 443 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:08 smithi155 bash[20805]: cluster 2024-02-18T01:28:07.558629+0000 mon.a (mon.0) 2231 : cluster [DBG] osdmap e360: 8 total, 8 up, 8 in 2024-02-18T01:28:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:08 smithi155 bash[20805]: cluster 2024-02-18T01:28:07.767657+0000 mgr.y (mgr.24475) 442 : cluster [DBG] pgmap v618: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-02-18T01:28:08.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:08 smithi155 bash[20805]: audit 2024-02-18T01:28:07.831769+0000 mgr.y (mgr.24475) 443 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[27669]: cluster 2024-02-18T01:28:08.573750+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[27669]: audit 2024-02-18T01:28:08.578996+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[27669]: audit 2024-02-18T01:28:08.583369+0000 mon.a (mon.0) 2233 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[19734]: cluster 2024-02-18T01:28:08.573750+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[19734]: audit 2024-02-18T01:28:08.578996+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:09 smithi105 bash[19734]: audit 2024-02-18T01:28:08.583369+0000 mon.a (mon.0) 2233 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:09 smithi155 bash[20805]: cluster 2024-02-18T01:28:08.573750+0000 mon.a (mon.0) 2232 : cluster [DBG] osdmap e361: 8 total, 8 up, 8 in 2024-02-18T01:28:09.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:09 smithi155 bash[20805]: audit 2024-02-18T01:28:08.578996+0000 mon.b (mon.1) 313 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:09.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:09 smithi155 bash[20805]: audit 2024-02-18T01:28:08.583369+0000 mon.a (mon.0) 2233 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.577079+0000 mon.a (mon.0) 2234 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: cluster 2024-02-18T01:28:09.577632+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.580469+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.630589+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.630719+0000 mon.a (mon.0) 2236 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.631919+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: audit 2024-02-18T01:28:09.631979+0000 mon.a (mon.0) 2237 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[27669]: cluster 2024-02-18T01:28:09.768495+0000 mgr.y (mgr.24475) 444 : cluster [DBG] pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.577079+0000 mon.a (mon.0) 2234 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: cluster 2024-02-18T01:28:09.577632+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.580469+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.630589+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.630719+0000 mon.a (mon.0) 2236 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.631919+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: audit 2024-02-18T01:28:09.631979+0000 mon.a (mon.0) 2237 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:10 smithi105 bash[19734]: cluster 2024-02-18T01:28:09.768495+0000 mgr.y (mgr.24475) 444 : cluster [DBG] pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.577079+0000 mon.a (mon.0) 2234 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: cluster 2024-02-18T01:28:09.577632+0000 mon.a (mon.0) 2235 : cluster [DBG] osdmap e362: 8 total, 8 up, 8 in 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.580469+0000 mon.b (mon.1) 314 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.630589+0000 mon.b (mon.1) 315 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.630719+0000 mon.a (mon.0) 2236 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.631919+0000 mon.b (mon.1) 316 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: audit 2024-02-18T01:28:09.631979+0000 mon.a (mon.0) 2237 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-53"}]: dispatch 2024-02-18T01:28:10.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:10 smithi155 bash[20805]: cluster 2024-02-18T01:28:09.768495+0000 mgr.y (mgr.24475) 444 : cluster [DBG] pgmap v621: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:11.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:11 smithi155 bash[20805]: cluster 2024-02-18T01:28:10.600296+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-18T01:28:11.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:11 smithi105 bash[19734]: cluster 2024-02-18T01:28:10.600296+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-18T01:28:11.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:11 smithi105 bash[27669]: cluster 2024-02-18T01:28:10.600296+0000 mon.a (mon.0) 2238 : cluster [DBG] osdmap e363: 8 total, 8 up, 8 in 2024-02-18T01:28:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[19734]: cluster 2024-02-18T01:28:11.606804+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[19734]: audit 2024-02-18T01:28:11.611012+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[19734]: audit 2024-02-18T01:28:11.616502+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[19734]: cluster 2024-02-18T01:28:11.769393+0000 mgr.y (mgr.24475) 445 : cluster [DBG] pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[19734]: audit 2024-02-18T01:28:12.611801+0000 mon.a (mon.0) 2241 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[27669]: cluster 2024-02-18T01:28:11.606804+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-18T01:28:12.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[27669]: audit 2024-02-18T01:28:11.611012+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[27669]: audit 2024-02-18T01:28:11.616502+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[27669]: cluster 2024-02-18T01:28:11.769393+0000 mgr.y (mgr.24475) 445 : cluster [DBG] pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:12.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:12 smithi105 bash[27669]: audit 2024-02-18T01:28:12.611801+0000 mon.a (mon.0) 2241 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:12.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:12 smithi155 bash[20805]: cluster 2024-02-18T01:28:11.606804+0000 mon.a (mon.0) 2239 : cluster [DBG] osdmap e364: 8 total, 8 up, 8 in 2024-02-18T01:28:12.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:12 smithi155 bash[20805]: audit 2024-02-18T01:28:11.611012+0000 mon.b (mon.1) 317 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:12 smithi155 bash[20805]: audit 2024-02-18T01:28:11.616502+0000 mon.a (mon.0) 2240 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:12.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:12 smithi155 bash[20805]: cluster 2024-02-18T01:28:11.769393+0000 mgr.y (mgr.24475) 445 : cluster [DBG] pgmap v624: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:12.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:12 smithi155 bash[20805]: audit 2024-02-18T01:28:12.611801+0000 mon.a (mon.0) 2241 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: cluster 2024-02-18T01:28:12.612126+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.661258+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.661347+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.662457+0000 mon.b (mon.1) 319 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.662513+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.683744+0000 mon.a (mon.0) 2245 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.995916+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.996395+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.997512+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.997928+0000 mon.a (mon.0) 2249 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.998937+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:12.999386+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.000660+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.000987+0000 mon.a (mon.0) 2253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: cephadm 2024-02-18T01:28:13.001731+0000 mgr.y (mgr.24475) 446 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3305M 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: cephadm 2024-02-18T01:28:13.002283+0000 mgr.y (mgr.24475) 447 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 3305M 2024-02-18T01:28:13.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.008465+0000 mon.a (mon.0) 2254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: cluster 2024-02-18T01:28:12.612126+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.661258+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.661347+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.662457+0000 mon.b (mon.1) 319 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.662513+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.683744+0000 mon.a (mon.0) 2245 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.995916+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.996395+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.997512+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.997928+0000 mon.a (mon.0) 2249 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.998937+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:12.999386+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.000660+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.000987+0000 mon.a (mon.0) 2253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: cephadm 2024-02-18T01:28:13.001731+0000 mgr.y (mgr.24475) 446 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3305M 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: cephadm 2024-02-18T01:28:13.002283+0000 mgr.y (mgr.24475) 447 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 3305M 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.008465+0000 mon.a (mon.0) 2254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.015514+0000 mon.a (mon.0) 2255 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.016714+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.017694+0000 mon.a (mon.0) 2257 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[27669]: audit 2024-02-18T01:28:13.026223+0000 mon.a (mon.0) 2258 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.994 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.015514+0000 mon.a (mon.0) 2255 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.016714+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.017694+0000 mon.a (mon.0) 2257 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:13 smithi105 bash[19734]: audit 2024-02-18T01:28:13.026223+0000 mon.a (mon.0) 2258 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: cluster 2024-02-18T01:28:12.612126+0000 mon.a (mon.0) 2242 : cluster [DBG] osdmap e365: 8 total, 8 up, 8 in 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.661258+0000 mon.b (mon.1) 318 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.661347+0000 mon.a (mon.0) 2243 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:13.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.662457+0000 mon.b (mon.1) 319 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.662513+0000 mon.a (mon.0) 2244 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-55"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.683744+0000 mon.a (mon.0) 2245 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.995916+0000 mon.a (mon.0) 2246 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.996395+0000 mon.a (mon.0) 2247 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.997512+0000 mon.a (mon.0) 2248 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.997928+0000 mon.a (mon.0) 2249 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.998937+0000 mon.a (mon.0) 2250 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:12.999386+0000 mon.a (mon.0) 2251 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.000660+0000 mon.a (mon.0) 2252 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.000987+0000 mon.a (mon.0) 2253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: cephadm 2024-02-18T01:28:13.001731+0000 mgr.y (mgr.24475) 446 : cephadm [INF] Adjusting osd_memory_target on smithi105 to 3305M 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: cephadm 2024-02-18T01:28:13.002283+0000 mgr.y (mgr.24475) 447 : cephadm [INF] Adjusting osd_memory_target on smithi155 to 3305M 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.008465+0000 mon.a (mon.0) 2254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.015514+0000 mon.a (mon.0) 2255 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:13.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.016714+0000 mon.a (mon.0) 2256 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:28:13.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.017694+0000 mon.a (mon.0) 2257 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:28:13.997 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:13 smithi155 bash[20805]: audit 2024-02-18T01:28:13.026223+0000 mon.a (mon.0) 2258 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:28:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:14 smithi105 bash[19734]: cluster 2024-02-18T01:28:13.636329+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-18T01:28:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:14 smithi105 bash[19734]: cluster 2024-02-18T01:28:13.769980+0000 mgr.y (mgr.24475) 448 : cluster [DBG] pgmap v627: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:14 smithi105 bash[27669]: cluster 2024-02-18T01:28:13.636329+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-18T01:28:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:14 smithi105 bash[27669]: cluster 2024-02-18T01:28:13.769980+0000 mgr.y (mgr.24475) 448 : cluster [DBG] pgmap v627: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:14 smithi155 bash[20805]: cluster 2024-02-18T01:28:13.636329+0000 mon.a (mon.0) 2259 : cluster [DBG] osdmap e366: 8 total, 8 up, 8 in 2024-02-18T01:28:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:14 smithi155 bash[20805]: cluster 2024-02-18T01:28:13.769980+0000 mgr.y (mgr.24475) 448 : cluster [DBG] pgmap v627: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[19734]: cluster 2024-02-18T01:28:14.652204+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-18T01:28:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[19734]: audit 2024-02-18T01:28:14.653752+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:15.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[19734]: audit 2024-02-18T01:28:14.654018+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[27669]: cluster 2024-02-18T01:28:14.652204+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-18T01:28:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[27669]: audit 2024-02-18T01:28:14.653752+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:15.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:15 smithi105 bash[27669]: audit 2024-02-18T01:28:14.654018+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:15 smithi155 bash[20805]: cluster 2024-02-18T01:28:14.652204+0000 mon.a (mon.0) 2260 : cluster [DBG] osdmap e367: 8 total, 8 up, 8 in 2024-02-18T01:28:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:15 smithi155 bash[20805]: audit 2024-02-18T01:28:14.653752+0000 mon.b (mon.1) 320 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:15.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:15 smithi155 bash[20805]: audit 2024-02-18T01:28:14.654018+0000 mon.a (mon.0) 2261 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:16] "GET /metrics HTTP/1.1" 200 188207 "" "Prometheus/2.33.4" 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.653570+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: cluster 2024-02-18T01:28:15.655181+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.657894+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.706400+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.706501+0000 mon.a (mon.0) 2264 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.707688+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: audit 2024-02-18T01:28:15.707705+0000 mon.b (mon.1) 323 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.006 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[19734]: cluster 2024-02-18T01:28:15.770818+0000 mgr.y (mgr.24475) 449 : cluster [DBG] pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.653570+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: cluster 2024-02-18T01:28:15.655181+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.657894+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.706400+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.706501+0000 mon.a (mon.0) 2264 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.707688+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: audit 2024-02-18T01:28:15.707705+0000 mon.b (mon.1) 323 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.007 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:16 smithi105 bash[27669]: cluster 2024-02-18T01:28:15.770818+0000 mgr.y (mgr.24475) 449 : cluster [DBG] pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.653570+0000 mon.a (mon.0) 2262 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: cluster 2024-02-18T01:28:15.655181+0000 mon.a (mon.0) 2263 : cluster [DBG] osdmap e368: 8 total, 8 up, 8 in 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.657894+0000 mon.b (mon.1) 321 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.706400+0000 mon.b (mon.1) 322 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.706501+0000 mon.a (mon.0) 2264 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.707688+0000 mon.a (mon.0) 2265 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: audit 2024-02-18T01:28:15.707705+0000 mon.b (mon.1) 323 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-57"}]: dispatch 2024-02-18T01:28:17.008 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:16 smithi155 bash[20805]: cluster 2024-02-18T01:28:15.770818+0000 mgr.y (mgr.24475) 449 : cluster [DBG] pgmap v630: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-02-18T01:28:17.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:17 smithi105 bash[19734]: cluster 2024-02-18T01:28:16.669956+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-18T01:28:17.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:17 smithi105 bash[27669]: cluster 2024-02-18T01:28:16.669956+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-18T01:28:17.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:17 smithi155 bash[20805]: cluster 2024-02-18T01:28:16.669956+0000 mon.a (mon.0) 2266 : cluster [DBG] osdmap e369: 8 total, 8 up, 8 in 2024-02-18T01:28:17.994 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[19734]: cluster 2024-02-18T01:28:17.692857+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-18T01:28:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[19734]: audit 2024-02-18T01:28:17.697758+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[19734]: audit 2024-02-18T01:28:17.703287+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[19734]: cluster 2024-02-18T01:28:17.771801+0000 mgr.y (mgr.24475) 450 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:18.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[19734]: audit 2024-02-18T01:28:17.843123+0000 mgr.y (mgr.24475) 451 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[27669]: cluster 2024-02-18T01:28:17.692857+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-18T01:28:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[27669]: audit 2024-02-18T01:28:17.697758+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[27669]: audit 2024-02-18T01:28:17.703287+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[27669]: cluster 2024-02-18T01:28:17.771801+0000 mgr.y (mgr.24475) 450 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:18.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:18 smithi105 bash[27669]: audit 2024-02-18T01:28:17.843123+0000 mgr.y (mgr.24475) 451 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:18 smithi155 bash[20805]: cluster 2024-02-18T01:28:17.692857+0000 mon.a (mon.0) 2267 : cluster [DBG] osdmap e370: 8 total, 8 up, 8 in 2024-02-18T01:28:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:18 smithi155 bash[20805]: audit 2024-02-18T01:28:17.697758+0000 mon.b (mon.1) 324 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:18 smithi155 bash[20805]: audit 2024-02-18T01:28:17.703287+0000 mon.a (mon.0) 2268 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:18 smithi155 bash[20805]: cluster 2024-02-18T01:28:17.771801+0000 mgr.y (mgr.24475) 450 : cluster [DBG] pgmap v633: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:18.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:18 smithi155 bash[20805]: audit 2024-02-18T01:28:17.843123+0000 mgr.y (mgr.24475) 451 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[27669]: audit 2024-02-18T01:28:18.697285+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[27669]: cluster 2024-02-18T01:28:18.697420+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-18T01:28:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[27669]: audit 2024-02-18T01:28:18.699941+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[27669]: audit 2024-02-18T01:28:18.703882+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:19.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[27669]: audit 2024-02-18T01:28:18.714320+0000 mon.a (mon.0) 2271 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:19.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[19734]: audit 2024-02-18T01:28:18.697285+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:19.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[19734]: cluster 2024-02-18T01:28:18.697420+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-18T01:28:19.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[19734]: audit 2024-02-18T01:28:18.699941+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:19.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[19734]: audit 2024-02-18T01:28:18.703882+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:19.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:19 smithi105 bash[19734]: audit 2024-02-18T01:28:18.714320+0000 mon.a (mon.0) 2271 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:19.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:19 smithi155 bash[20805]: audit 2024-02-18T01:28:18.697285+0000 mon.a (mon.0) 2269 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:19.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:19 smithi155 bash[20805]: cluster 2024-02-18T01:28:18.697420+0000 mon.a (mon.0) 2270 : cluster [DBG] osdmap e371: 8 total, 8 up, 8 in 2024-02-18T01:28:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:19 smithi155 bash[20805]: audit 2024-02-18T01:28:18.699941+0000 mon.b (mon.1) 325 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:19 smithi155 bash[20805]: audit 2024-02-18T01:28:18.703882+0000 mon.b (mon.1) 326 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:19.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:19 smithi155 bash[20805]: audit 2024-02-18T01:28:18.714320+0000 mon.a (mon.0) 2271 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: audit 2024-02-18T01:28:19.718198+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: cluster 2024-02-18T01:28:19.718291+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: audit 2024-02-18T01:28:19.766378+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: audit 2024-02-18T01:28:19.766716+0000 mon.a (mon.0) 2274 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: audit 2024-02-18T01:28:19.768124+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: audit 2024-02-18T01:28:19.768142+0000 mon.b (mon.1) 328 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[19734]: cluster 2024-02-18T01:28:19.772557+0000 mgr.y (mgr.24475) 452 : cluster [DBG] pgmap v636: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:20.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: audit 2024-02-18T01:28:19.718198+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: cluster 2024-02-18T01:28:19.718291+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: audit 2024-02-18T01:28:19.766378+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: audit 2024-02-18T01:28:19.766716+0000 mon.a (mon.0) 2274 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: audit 2024-02-18T01:28:19.768124+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: audit 2024-02-18T01:28:19.768142+0000 mon.b (mon.1) 328 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:20 smithi105 bash[27669]: cluster 2024-02-18T01:28:19.772557+0000 mgr.y (mgr.24475) 452 : cluster [DBG] pgmap v636: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:20.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: audit 2024-02-18T01:28:19.718198+0000 mon.a (mon.0) 2272 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: cluster 2024-02-18T01:28:19.718291+0000 mon.a (mon.0) 2273 : cluster [DBG] osdmap e372: 8 total, 8 up, 8 in 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: audit 2024-02-18T01:28:19.766378+0000 mon.b (mon.1) 327 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: audit 2024-02-18T01:28:19.766716+0000 mon.a (mon.0) 2274 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: audit 2024-02-18T01:28:19.768124+0000 mon.a (mon.0) 2275 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: audit 2024-02-18T01:28:19.768142+0000 mon.b (mon.1) 328 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-59"}]: dispatch 2024-02-18T01:28:20.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:20 smithi155 bash[20805]: cluster 2024-02-18T01:28:19.772557+0000 mgr.y (mgr.24475) 452 : cluster [DBG] pgmap v636: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-02-18T01:28:21.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:21 smithi105 bash[19734]: cluster 2024-02-18T01:28:20.737697+0000 mon.a (mon.0) 2276 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-18T01:28:21.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:21 smithi105 bash[27669]: cluster 2024-02-18T01:28:20.737697+0000 mon.a (mon.0) 2276 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-18T01:28:21.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:21 smithi155 bash[20805]: cluster 2024-02-18T01:28:20.737697+0000 mon.a (mon.0) 2276 : cluster [DBG] osdmap e373: 8 total, 8 up, 8 in 2024-02-18T01:28:21.994 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[19734]: cluster 2024-02-18T01:28:21.752346+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-18T01:28:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[19734]: audit 2024-02-18T01:28:21.762298+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[19734]: audit 2024-02-18T01:28:21.762589+0000 mon.a (mon.0) 2278 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[19734]: cluster 2024-02-18T01:28:21.773026+0000 mgr.y (mgr.24475) 453 : cluster [DBG] pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:23.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[27669]: cluster 2024-02-18T01:28:21.752346+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-18T01:28:23.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[27669]: audit 2024-02-18T01:28:21.762298+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[27669]: audit 2024-02-18T01:28:21.762589+0000 mon.a (mon.0) 2278 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:22 smithi105 bash[27669]: cluster 2024-02-18T01:28:21.773026+0000 mgr.y (mgr.24475) 453 : cluster [DBG] pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:22 smithi155 bash[20805]: cluster 2024-02-18T01:28:21.752346+0000 mon.a (mon.0) 2277 : cluster [DBG] osdmap e374: 8 total, 8 up, 8 in 2024-02-18T01:28:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:22 smithi155 bash[20805]: audit 2024-02-18T01:28:21.762298+0000 mon.b (mon.1) 329 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:22 smithi155 bash[20805]: audit 2024-02-18T01:28:21.762589+0000 mon.a (mon.0) 2278 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:28:23.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:22 smithi155 bash[20805]: cluster 2024-02-18T01:28:21.773026+0000 mgr.y (mgr.24475) 453 : cluster [DBG] pgmap v639: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:28:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[19734]: audit 2024-02-18T01:28:22.758913+0000 mon.a (mon.0) 2279 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[19734]: cluster 2024-02-18T01:28:22.759024+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-18T01:28:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[19734]: audit 2024-02-18T01:28:22.763063+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[19734]: audit 2024-02-18T01:28:22.768392+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[19734]: audit 2024-02-18T01:28:22.768461+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:24.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[27669]: audit 2024-02-18T01:28:22.758913+0000 mon.a (mon.0) 2279 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:24.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[27669]: cluster 2024-02-18T01:28:22.759024+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-18T01:28:24.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[27669]: audit 2024-02-18T01:28:22.763063+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:24.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[27669]: audit 2024-02-18T01:28:22.768392+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:24.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:23 smithi105 bash[27669]: audit 2024-02-18T01:28:22.768461+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:23 smithi155 bash[20805]: audit 2024-02-18T01:28:22.758913+0000 mon.a (mon.0) 2279 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:28:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:23 smithi155 bash[20805]: cluster 2024-02-18T01:28:22.759024+0000 mon.a (mon.0) 2280 : cluster [DBG] osdmap e375: 8 total, 8 up, 8 in 2024-02-18T01:28:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:23 smithi155 bash[20805]: audit 2024-02-18T01:28:22.763063+0000 mon.b (mon.1) 330 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:28:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:23 smithi155 bash[20805]: audit 2024-02-18T01:28:22.768392+0000 mon.b (mon.1) 331 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:24.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:23 smithi155 bash[20805]: audit 2024-02-18T01:28:22.768461+0000 mon.a (mon.0) 2281 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[27669]: audit 2024-02-18T01:28:23.762065+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[27669]: cluster 2024-02-18T01:28:23.762226+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[27669]: cluster 2024-02-18T01:28:23.773666+0000 mgr.y (mgr.24475) 454 : cluster [DBG] pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[19734]: audit 2024-02-18T01:28:23.762065+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[19734]: cluster 2024-02-18T01:28:23.762226+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-18T01:28:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:24 smithi105 bash[19734]: cluster 2024-02-18T01:28:23.773666+0000 mgr.y (mgr.24475) 454 : cluster [DBG] pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:24 smithi155 bash[20805]: audit 2024-02-18T01:28:23.762065+0000 mon.a (mon.0) 2282 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:28:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:24 smithi155 bash[20805]: cluster 2024-02-18T01:28:23.762226+0000 mon.a (mon.0) 2283 : cluster [DBG] osdmap e376: 8 total, 8 up, 8 in 2024-02-18T01:28:25.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:24 smithi155 bash[20805]: cluster 2024-02-18T01:28:23.773666+0000 mgr.y (mgr.24475) 454 : cluster [DBG] pgmap v642: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:28:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:25 smithi105 bash[19734]: cluster 2024-02-18T01:28:24.783107+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-18T01:28:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:25 smithi105 bash[27669]: cluster 2024-02-18T01:28:24.783107+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-18T01:28:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:25 smithi155 bash[20805]: cluster 2024-02-18T01:28:24.783107+0000 mon.a (mon.0) 2284 : cluster [DBG] osdmap e377: 8 total, 8 up, 8 in 2024-02-18T01:28:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:26 smithi105 bash[19734]: cluster 2024-02-18T01:28:25.774499+0000 mgr.y (mgr.24475) 455 : cluster [DBG] pgmap v644: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1017 B/s rd, 508 B/s wr, 1 op/s 2024-02-18T01:28:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:26 smithi105 bash[19734]: cluster 2024-02-18T01:28:25.785972+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-18T01:28:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:26] "GET /metrics HTTP/1.1" 200 185759 "" "Prometheus/2.33.4" 2024-02-18T01:28:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:26 smithi105 bash[27669]: cluster 2024-02-18T01:28:25.774499+0000 mgr.y (mgr.24475) 455 : cluster [DBG] pgmap v644: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1017 B/s rd, 508 B/s wr, 1 op/s 2024-02-18T01:28:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:26 smithi105 bash[27669]: cluster 2024-02-18T01:28:25.785972+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-18T01:28:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:26 smithi155 bash[20805]: cluster 2024-02-18T01:28:25.774499+0000 mgr.y (mgr.24475) 455 : cluster [DBG] pgmap v644: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1017 B/s rd, 508 B/s wr, 1 op/s 2024-02-18T01:28:27.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:26 smithi155 bash[20805]: cluster 2024-02-18T01:28:25.785972+0000 mon.a (mon.0) 2285 : cluster [DBG] osdmap e378: 8 total, 8 up, 8 in 2024-02-18T01:28:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:27 smithi105 bash[27669]: cluster 2024-02-18T01:28:26.798040+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-18T01:28:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:27 smithi105 bash[19734]: cluster 2024-02-18T01:28:26.798040+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-18T01:28:28.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:28.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:27 smithi155 bash[20805]: cluster 2024-02-18T01:28:26.798040+0000 mon.a (mon.0) 2286 : cluster [DBG] osdmap e379: 8 total, 8 up, 8 in 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[19734]: cluster 2024-02-18T01:28:27.776811+0000 mgr.y (mgr.24475) 456 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[19734]: cluster 2024-02-18T01:28:27.817328+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[19734]: audit 2024-02-18T01:28:27.854399+0000 mgr.y (mgr.24475) 457 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[27669]: cluster 2024-02-18T01:28:27.776811+0000 mgr.y (mgr.24475) 456 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[27669]: cluster 2024-02-18T01:28:27.817328+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-18T01:28:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:28 smithi105 bash[27669]: audit 2024-02-18T01:28:27.854399+0000 mgr.y (mgr.24475) 457 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:28 smithi155 bash[20805]: cluster 2024-02-18T01:28:27.776811+0000 mgr.y (mgr.24475) 456 : cluster [DBG] pgmap v647: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-02-18T01:28:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:28 smithi155 bash[20805]: cluster 2024-02-18T01:28:27.817328+0000 mon.a (mon.0) 2287 : cluster [DBG] osdmap e380: 8 total, 8 up, 8 in 2024-02-18T01:28:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:28 smithi155 bash[20805]: audit 2024-02-18T01:28:27.854399+0000 mgr.y (mgr.24475) 457 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:30 smithi105 bash[19734]: cluster 2024-02-18T01:28:29.777666+0000 mgr.y (mgr.24475) 458 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-18T01:28:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:30 smithi105 bash[27669]: cluster 2024-02-18T01:28:29.777666+0000 mgr.y (mgr.24475) 458 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-18T01:28:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:30 smithi155 bash[20805]: cluster 2024-02-18T01:28:29.777666+0000 mgr.y (mgr.24475) 458 : cluster [DBG] pgmap v649: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 2.2 KiB/s wr, 6 op/s 2024-02-18T01:28:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:32 smithi105 bash[19734]: cluster 2024-02-18T01:28:31.778449+0000 mgr.y (mgr.24475) 459 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T01:28:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:32 smithi105 bash[27669]: cluster 2024-02-18T01:28:31.778449+0000 mgr.y (mgr.24475) 459 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T01:28:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:32 smithi155 bash[20805]: cluster 2024-02-18T01:28:31.778449+0000 mgr.y (mgr.24475) 459 : cluster [DBG] pgmap v650: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.8 KiB/s wr, 4 op/s 2024-02-18T01:28:35.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:34 smithi105 bash[19734]: cluster 2024-02-18T01:28:33.780782+0000 mgr.y (mgr.24475) 460 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-18T01:28:35.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:34 smithi105 bash[19734]: cluster 2024-02-18T01:28:34.100389+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-18T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:34 smithi105 bash[27669]: cluster 2024-02-18T01:28:33.780782+0000 mgr.y (mgr.24475) 460 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-18T01:28:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:34 smithi105 bash[27669]: cluster 2024-02-18T01:28:34.100389+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-18T01:28:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:34 smithi155 bash[20805]: cluster 2024-02-18T01:28:33.780782+0000 mgr.y (mgr.24475) 460 : cluster [DBG] pgmap v651: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-02-18T01:28:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:34 smithi155 bash[20805]: cluster 2024-02-18T01:28:34.100389+0000 mon.a (mon.0) 2288 : cluster [DBG] osdmap e381: 8 total, 8 up, 8 in 2024-02-18T01:28:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:36] "GET /metrics HTTP/1.1" 200 188326 "" "Prometheus/2.33.4" 2024-02-18T01:28:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[19734]: cluster 2024-02-18T01:28:35.781681+0000 mgr.y (mgr.24475) 461 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 0 op/s 2024-02-18T01:28:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[19734]: audit 2024-02-18T01:28:36.686504+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]: dispatch 2024-02-18T01:28:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[19734]: audit 2024-02-18T01:28:36.686735+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-18T01:28:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[27669]: cluster 2024-02-18T01:28:35.781681+0000 mgr.y (mgr.24475) 461 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 0 op/s 2024-02-18T01:28:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[27669]: audit 2024-02-18T01:28:36.686504+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]: dispatch 2024-02-18T01:28:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:36 smithi105 bash[27669]: audit 2024-02-18T01:28:36.686735+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-18T01:28:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:36 smithi155 bash[20805]: cluster 2024-02-18T01:28:35.781681+0000 mgr.y (mgr.24475) 461 : cluster [DBG] pgmap v653: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 B/s wr, 0 op/s 2024-02-18T01:28:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:36 smithi155 bash[20805]: audit 2024-02-18T01:28:36.686504+0000 mon.a (mon.0) 2289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]: dispatch 2024-02-18T01:28:37.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:36 smithi155 bash[20805]: audit 2024-02-18T01:28:36.686735+0000 mon.a (mon.0) 2290 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]: dispatch 2024-02-18T01:28:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[19734]: audit 2024-02-18T01:28:36.840768+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]': finished 2024-02-18T01:28:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[19734]: audit 2024-02-18T01:28:36.841010+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-18T01:28:38.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[19734]: cluster 2024-02-18T01:28:36.841210+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-18T01:28:38.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[27669]: audit 2024-02-18T01:28:36.840768+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]': finished 2024-02-18T01:28:38.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[27669]: audit 2024-02-18T01:28:36.841010+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-18T01:28:38.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:37 smithi105 bash[27669]: cluster 2024-02-18T01:28:36.841210+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-18T01:28:38.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:38.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:37 smithi155 bash[20805]: audit 2024-02-18T01:28:36.840768+0000 mon.a (mon.0) 2291 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.2", "id": [1, 3]}]': finished 2024-02-18T01:28:38.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:37 smithi155 bash[20805]: audit 2024-02-18T01:28:36.841010+0000 mon.a (mon.0) 2292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "63.16", "id": [1, 3]}]': finished 2024-02-18T01:28:38.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:37 smithi155 bash[20805]: cluster 2024-02-18T01:28:36.841210+0000 mon.a (mon.0) 2293 : cluster [DBG] osdmap e382: 8 total, 8 up, 8 in 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[27669]: cluster 2024-02-18T01:28:37.783820+0000 mgr.y (mgr.24475) 462 : cluster [DBG] pgmap v655: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[27669]: audit 2024-02-18T01:28:37.863393+0000 mgr.y (mgr.24475) 463 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[27669]: cluster 2024-02-18T01:28:37.863716+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[19734]: cluster 2024-02-18T01:28:37.783820+0000 mgr.y (mgr.24475) 462 : cluster [DBG] pgmap v655: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[19734]: audit 2024-02-18T01:28:37.863393+0000 mgr.y (mgr.24475) 463 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:38 smithi105 bash[19734]: cluster 2024-02-18T01:28:37.863716+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-18T01:28:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:38 smithi155 bash[20805]: cluster 2024-02-18T01:28:37.783820+0000 mgr.y (mgr.24475) 462 : cluster [DBG] pgmap v655: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 op/s 2024-02-18T01:28:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:38 smithi155 bash[20805]: audit 2024-02-18T01:28:37.863393+0000 mgr.y (mgr.24475) 463 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:38 smithi155 bash[20805]: cluster 2024-02-18T01:28:37.863716+0000 mon.a (mon.0) 2294 : cluster [DBG] osdmap e383: 8 total, 8 up, 8 in 2024-02-18T01:28:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:40 smithi105 bash[19734]: cluster 2024-02-18T01:28:39.784611+0000 mgr.y (mgr.24475) 464 : cluster [DBG] pgmap v657: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:28:41.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:40 smithi105 bash[27669]: cluster 2024-02-18T01:28:39.784611+0000 mgr.y (mgr.24475) 464 : cluster [DBG] pgmap v657: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:28:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:40 smithi155 bash[20805]: cluster 2024-02-18T01:28:39.784611+0000 mgr.y (mgr.24475) 464 : cluster [DBG] pgmap v657: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:28:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:43.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:42 smithi105 bash[27669]: cluster 2024-02-18T01:28:41.785490+0000 mgr.y (mgr.24475) 465 : cluster [DBG] pgmap v658: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-02-18T01:28:43.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:42 smithi105 bash[19734]: cluster 2024-02-18T01:28:41.785490+0000 mgr.y (mgr.24475) 465 : cluster [DBG] pgmap v658: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-02-18T01:28:43.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:42 smithi155 bash[20805]: cluster 2024-02-18T01:28:41.785490+0000 mgr.y (mgr.24475) 465 : cluster [DBG] pgmap v658: 228 pgs: 2 peering, 226 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 665 B/s rd, 0 op/s 2024-02-18T01:28:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:44 smithi105 bash[27669]: cluster 2024-02-18T01:28:43.787728+0000 mgr.y (mgr.24475) 466 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:45.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:44 smithi105 bash[27669]: cluster 2024-02-18T01:28:44.101082+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-18T01:28:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:44 smithi105 bash[19734]: cluster 2024-02-18T01:28:43.787728+0000 mgr.y (mgr.24475) 466 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:44 smithi105 bash[19734]: cluster 2024-02-18T01:28:44.101082+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-18T01:28:45.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:44 smithi155 bash[20805]: cluster 2024-02-18T01:28:43.787728+0000 mgr.y (mgr.24475) 466 : cluster [DBG] pgmap v659: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:45.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:44 smithi155 bash[20805]: cluster 2024-02-18T01:28:44.101082+0000 mon.a (mon.0) 2295 : cluster [DBG] osdmap e384: 8 total, 8 up, 8 in 2024-02-18T01:28:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:46 smithi105 bash[19734]: cluster 2024-02-18T01:28:45.788574+0000 mgr.y (mgr.24475) 467 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.9 KiB/s wr, 3 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:46 smithi105 bash[27669]: cluster 2024-02-18T01:28:45.788574+0000 mgr.y (mgr.24475) 467 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.9 KiB/s wr, 3 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:46] "GET /metrics HTTP/1.1" 200 188326 "" "Prometheus/2.33.4" 2024-02-18T01:28:47.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:46 smithi155 bash[20805]: cluster 2024-02-18T01:28:45.788574+0000 mgr.y (mgr.24475) 467 : cluster [DBG] pgmap v661: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.9 KiB/s wr, 3 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:48.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[19734]: cluster 2024-02-18T01:28:47.790727+0000 mgr.y (mgr.24475) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 9 B/s, 4 objects/s recovering 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[19734]: audit 2024-02-18T01:28:47.869613+0000 mgr.y (mgr.24475) 469 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[19734]: cluster 2024-02-18T01:28:47.892747+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[27669]: cluster 2024-02-18T01:28:47.790727+0000 mgr.y (mgr.24475) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 9 B/s, 4 objects/s recovering 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[27669]: audit 2024-02-18T01:28:47.869613+0000 mgr.y (mgr.24475) 469 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:48 smithi105 bash[27669]: cluster 2024-02-18T01:28:47.892747+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-18T01:28:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:48 smithi155 bash[20805]: cluster 2024-02-18T01:28:47.790727+0000 mgr.y (mgr.24475) 468 : cluster [DBG] pgmap v662: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 3 op/s; 9 B/s, 4 objects/s recovering 2024-02-18T01:28:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:48 smithi155 bash[20805]: audit 2024-02-18T01:28:47.869613+0000 mgr.y (mgr.24475) 469 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:48 smithi155 bash[20805]: cluster 2024-02-18T01:28:47.892747+0000 mon.a (mon.0) 2296 : cluster [DBG] osdmap e385: 8 total, 8 up, 8 in 2024-02-18T01:28:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:50 smithi105 bash[19734]: cluster 2024-02-18T01:28:49.791569+0000 mgr.y (mgr.24475) 470 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:51.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:50 smithi105 bash[27669]: cluster 2024-02-18T01:28:49.791569+0000 mgr.y (mgr.24475) 470 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:50 smithi155 bash[20805]: cluster 2024-02-18T01:28:49.791569+0000 mgr.y (mgr.24475) 470 : cluster [DBG] pgmap v664: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.9 KiB/s wr, 4 op/s; 12 B/s, 5 objects/s recovering 2024-02-18T01:28:52.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:28:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:28:53.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:52 smithi105 bash[19734]: cluster 2024-02-18T01:28:51.792489+0000 mgr.y (mgr.24475) 471 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:28:53.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:52 smithi105 bash[27669]: cluster 2024-02-18T01:28:51.792489+0000 mgr.y (mgr.24475) 471 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:28:53.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:52 smithi155 bash[20805]: cluster 2024-02-18T01:28:51.792489+0000 mgr.y (mgr.24475) 471 : cluster [DBG] pgmap v665: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:28:55.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:54 smithi105 bash[19734]: cluster 2024-02-18T01:28:53.794738+0000 mgr.y (mgr.24475) 472 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:55.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:54 smithi105 bash[19734]: cluster 2024-02-18T01:28:54.104840+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-18T01:28:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:54 smithi105 bash[27669]: cluster 2024-02-18T01:28:53.794738+0000 mgr.y (mgr.24475) 472 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:54 smithi105 bash[27669]: cluster 2024-02-18T01:28:54.104840+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-18T01:28:55.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:54 smithi155 bash[20805]: cluster 2024-02-18T01:28:53.794738+0000 mgr.y (mgr.24475) 472 : cluster [DBG] pgmap v666: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:55.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:54 smithi155 bash[20805]: cluster 2024-02-18T01:28:54.104840+0000 mon.a (mon.0) 2297 : cluster [DBG] osdmap e386: 8 total, 8 up, 8 in 2024-02-18T01:28:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:56 smithi105 bash[27669]: cluster 2024-02-18T01:28:55.795518+0000 mgr.y (mgr.24475) 473 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:28:56.992 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:28:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:28:56] "GET /metrics HTTP/1.1" 200 188336 "" "Prometheus/2.33.4" 2024-02-18T01:28:56.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:56 smithi105 bash[19734]: cluster 2024-02-18T01:28:55.795518+0000 mgr.y (mgr.24475) 473 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:28:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:56 smithi155 bash[20805]: cluster 2024-02-18T01:28:55.795518+0000 mgr.y (mgr.24475) 473 : cluster [DBG] pgmap v668: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:28:58.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:28:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: cluster 2024-02-18T01:28:57.797631+0000 mgr.y (mgr.24475) 474 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: audit 2024-02-18T01:28:57.880871+0000 mgr.y (mgr.24475) 475 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: audit 2024-02-18T01:28:57.929006+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: audit 2024-02-18T01:28:57.929143+0000 mon.a (mon.0) 2298 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: audit 2024-02-18T01:28:57.930603+0000 mon.b (mon.1) 333 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[19734]: audit 2024-02-18T01:28:57.930675+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:28:59.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: cluster 2024-02-18T01:28:57.797631+0000 mgr.y (mgr.24475) 474 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: audit 2024-02-18T01:28:57.880871+0000 mgr.y (mgr.24475) 475 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: audit 2024-02-18T01:28:57.929006+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: audit 2024-02-18T01:28:57.929143+0000 mon.a (mon.0) 2298 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: audit 2024-02-18T01:28:57.930603+0000 mon.b (mon.1) 333 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:28:59.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:58 smithi105 bash[27669]: audit 2024-02-18T01:28:57.930675+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:28:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: cluster 2024-02-18T01:28:57.797631+0000 mgr.y (mgr.24475) 474 : cluster [DBG] pgmap v669: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:28:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: audit 2024-02-18T01:28:57.880871+0000 mgr.y (mgr.24475) 475 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:28:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: audit 2024-02-18T01:28:57.929006+0000 mon.b (mon.1) 332 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: audit 2024-02-18T01:28:57.929143+0000 mon.a (mon.0) 2298 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:28:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: audit 2024-02-18T01:28:57.930603+0000 mon.b (mon.1) 333 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:28:59.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:58 smithi155 bash[20805]: audit 2024-02-18T01:28:57.930675+0000 mon.a (mon.0) 2299 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-61"}]: dispatch 2024-02-18T01:29:00.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:28:59 smithi105 bash[27669]: cluster 2024-02-18T01:28:58.926280+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-18T01:29:00.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:28:59 smithi105 bash[19734]: cluster 2024-02-18T01:28:58.926280+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-18T01:29:00.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:28:59 smithi155 bash[20805]: cluster 2024-02-18T01:28:58.926280+0000 mon.a (mon.0) 2300 : cluster [DBG] osdmap e387: 8 total, 8 up, 8 in 2024-02-18T01:29:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[19734]: cluster 2024-02-18T01:28:59.798430+0000 mgr.y (mgr.24475) 476 : cluster [DBG] pgmap v671: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[19734]: cluster 2024-02-18T01:28:59.946828+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-18T01:29:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[19734]: audit 2024-02-18T01:28:59.948673+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:01.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[19734]: audit 2024-02-18T01:28:59.951163+0000 mon.a (mon.0) 2302 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[27669]: cluster 2024-02-18T01:28:59.798430+0000 mgr.y (mgr.24475) 476 : cluster [DBG] pgmap v671: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[27669]: cluster 2024-02-18T01:28:59.946828+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-18T01:29:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[27669]: audit 2024-02-18T01:28:59.948673+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:01.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:00 smithi105 bash[27669]: audit 2024-02-18T01:28:59.951163+0000 mon.a (mon.0) 2302 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:00 smithi155 bash[20805]: cluster 2024-02-18T01:28:59.798430+0000 mgr.y (mgr.24475) 476 : cluster [DBG] pgmap v671: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:00 smithi155 bash[20805]: cluster 2024-02-18T01:28:59.946828+0000 mon.a (mon.0) 2301 : cluster [DBG] osdmap e388: 8 total, 8 up, 8 in 2024-02-18T01:29:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:00 smithi155 bash[20805]: audit 2024-02-18T01:28:59.948673+0000 mon.b (mon.1) 334 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:01.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:00 smithi155 bash[20805]: audit 2024-02-18T01:28:59.951163+0000 mon.a (mon.0) 2302 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[19734]: audit 2024-02-18T01:29:00.946773+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[19734]: cluster 2024-02-18T01:29:00.947532+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[19734]: audit 2024-02-18T01:29:00.957330+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[19734]: audit 2024-02-18T01:29:00.962277+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[19734]: audit 2024-02-18T01:29:00.963001+0000 mon.a (mon.0) 2305 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[27669]: audit 2024-02-18T01:29:00.946773+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:02.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[27669]: cluster 2024-02-18T01:29:00.947532+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-18T01:29:02.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[27669]: audit 2024-02-18T01:29:00.957330+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:02.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[27669]: audit 2024-02-18T01:29:00.962277+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:02.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:01 smithi105 bash[27669]: audit 2024-02-18T01:29:00.963001+0000 mon.a (mon.0) 2305 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[20805]: audit 2024-02-18T01:29:00.946773+0000 mon.a (mon.0) 2303 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[20805]: cluster 2024-02-18T01:29:00.947532+0000 mon.a (mon.0) 2304 : cluster [DBG] osdmap e389: 8 total, 8 up, 8 in 2024-02-18T01:29:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[20805]: audit 2024-02-18T01:29:00.957330+0000 mon.b (mon.1) 335 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[20805]: audit 2024-02-18T01:29:00.962277+0000 mon.b (mon.1) 336 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:02.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:01 smithi155 bash[20805]: audit 2024-02-18T01:29:00.963001+0000 mon.a (mon.0) 2305 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[27669]: cluster 2024-02-18T01:29:01.799235+0000 mgr.y (mgr.24475) 477 : cluster [DBG] pgmap v674: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[27669]: audit 2024-02-18T01:29:01.967152+0000 mon.a (mon.0) 2306 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[27669]: cluster 2024-02-18T01:29:01.967318+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[19734]: cluster 2024-02-18T01:29:01.799235+0000 mgr.y (mgr.24475) 477 : cluster [DBG] pgmap v674: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[19734]: audit 2024-02-18T01:29:01.967152+0000 mon.a (mon.0) 2306 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:02 smithi105 bash[19734]: cluster 2024-02-18T01:29:01.967318+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-18T01:29:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:02 smithi155 bash[20805]: cluster 2024-02-18T01:29:01.799235+0000 mgr.y (mgr.24475) 477 : cluster [DBG] pgmap v674: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:02 smithi155 bash[20805]: audit 2024-02-18T01:29:01.967152+0000 mon.a (mon.0) 2306 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:02 smithi155 bash[20805]: cluster 2024-02-18T01:29:01.967318+0000 mon.a (mon.0) 2307 : cluster [DBG] osdmap e390: 8 total, 8 up, 8 in 2024-02-18T01:29:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:03 smithi105 bash[19734]: cluster 2024-02-18T01:29:02.975647+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-18T01:29:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:03 smithi105 bash[27669]: cluster 2024-02-18T01:29:02.975647+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-18T01:29:04.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:03 smithi155 bash[20805]: cluster 2024-02-18T01:29:02.975647+0000 mon.a (mon.0) 2308 : cluster [DBG] osdmap e391: 8 total, 8 up, 8 in 2024-02-18T01:29:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:04 smithi105 bash[19734]: cluster 2024-02-18T01:29:03.800021+0000 mgr.y (mgr.24475) 478 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:04 smithi105 bash[19734]: cluster 2024-02-18T01:29:03.986666+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-18T01:29:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:04 smithi105 bash[27669]: cluster 2024-02-18T01:29:03.800021+0000 mgr.y (mgr.24475) 478 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:05 smithi105 bash[27669]: cluster 2024-02-18T01:29:03.986666+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-18T01:29:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:04 smithi155 bash[20805]: cluster 2024-02-18T01:29:03.800021+0000 mgr.y (mgr.24475) 478 : cluster [DBG] pgmap v677: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:04 smithi155 bash[20805]: cluster 2024-02-18T01:29:03.986666+0000 mon.a (mon.0) 2309 : cluster [DBG] osdmap e392: 8 total, 8 up, 8 in 2024-02-18T01:29:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:06 smithi105 bash[27669]: cluster 2024-02-18T01:29:04.993184+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-18T01:29:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:06 smithi105 bash[19734]: cluster 2024-02-18T01:29:04.993184+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-18T01:29:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:06 smithi155 bash[20805]: cluster 2024-02-18T01:29:04.993184+0000 mon.a (mon.0) 2310 : cluster [DBG] osdmap e393: 8 total, 8 up, 8 in 2024-02-18T01:29:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:06] "GET /metrics HTTP/1.1" 200 188457 "" "Prometheus/2.33.4" 2024-02-18T01:29:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:07 smithi105 bash[27669]: cluster 2024-02-18T01:29:05.800524+0000 mgr.y (mgr.24475) 479 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:07 smithi105 bash[19734]: cluster 2024-02-18T01:29:05.800524+0000 mgr.y (mgr.24475) 479 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:07 smithi155 bash[20805]: cluster 2024-02-18T01:29:05.800524+0000 mgr.y (mgr.24475) 479 : cluster [DBG] pgmap v680: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-02-18T01:29:08.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:09 smithi105 bash[19734]: cluster 2024-02-18T01:29:07.802731+0000 mgr.y (mgr.24475) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-18T01:29:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:09 smithi105 bash[19734]: audit 2024-02-18T01:29:07.892193+0000 mgr.y (mgr.24475) 481 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:09 smithi105 bash[27669]: cluster 2024-02-18T01:29:07.802731+0000 mgr.y (mgr.24475) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-18T01:29:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:09 smithi105 bash[27669]: audit 2024-02-18T01:29:07.892193+0000 mgr.y (mgr.24475) 481 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:09 smithi155 bash[20805]: cluster 2024-02-18T01:29:07.802731+0000 mgr.y (mgr.24475) 480 : cluster [DBG] pgmap v681: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.4 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2024-02-18T01:29:09.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:09 smithi155 bash[20805]: audit 2024-02-18T01:29:07.892193+0000 mgr.y (mgr.24475) 481 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:10 smithi105 bash[19734]: cluster 2024-02-18T01:29:09.385574+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-18T01:29:10.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:10 smithi105 bash[19734]: cluster 2024-02-18T01:29:09.803591+0000 mgr.y (mgr.24475) 482 : cluster [DBG] pgmap v683: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-18T01:29:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:10 smithi105 bash[27669]: cluster 2024-02-18T01:29:09.385574+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-18T01:29:10.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:10 smithi105 bash[27669]: cluster 2024-02-18T01:29:09.803591+0000 mgr.y (mgr.24475) 482 : cluster [DBG] pgmap v683: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-18T01:29:10.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:10 smithi155 bash[20805]: cluster 2024-02-18T01:29:09.385574+0000 mon.a (mon.0) 2311 : cluster [DBG] osdmap e394: 8 total, 8 up, 8 in 2024-02-18T01:29:10.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:10 smithi155 bash[20805]: cluster 2024-02-18T01:29:09.803591+0000 mgr.y (mgr.24475) 482 : cluster [DBG] pgmap v683: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-02-18T01:29:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:12 smithi105 bash[27669]: cluster 2024-02-18T01:29:11.804443+0000 mgr.y (mgr.24475) 483 : cluster [DBG] pgmap v684: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 523 B/s wr, 3 op/s 2024-02-18T01:29:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:12 smithi105 bash[19734]: cluster 2024-02-18T01:29:11.804443+0000 mgr.y (mgr.24475) 483 : cluster [DBG] pgmap v684: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 523 B/s wr, 3 op/s 2024-02-18T01:29:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:12 smithi155 bash[20805]: cluster 2024-02-18T01:29:11.804443+0000 mgr.y (mgr.24475) 483 : cluster [DBG] pgmap v684: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 523 B/s wr, 3 op/s 2024-02-18T01:29:14.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:13 smithi105 bash[19734]: audit 2024-02-18T01:29:13.034217+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:29:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:13 smithi105 bash[27669]: audit 2024-02-18T01:29:13.034217+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:29:14.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:13 smithi155 bash[20805]: audit 2024-02-18T01:29:13.034217+0000 mon.a (mon.0) 2312 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:29:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:14 smithi105 bash[19734]: cluster 2024-02-18T01:29:13.806687+0000 mgr.y (mgr.24475) 484 : cluster [DBG] pgmap v685: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 464 B/s wr, 3 op/s 2024-02-18T01:29:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:14 smithi105 bash[27669]: cluster 2024-02-18T01:29:13.806687+0000 mgr.y (mgr.24475) 484 : cluster [DBG] pgmap v685: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 464 B/s wr, 3 op/s 2024-02-18T01:29:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:14 smithi155 bash[20805]: cluster 2024-02-18T01:29:13.806687+0000 mgr.y (mgr.24475) 484 : cluster [DBG] pgmap v685: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.3 KiB/s rd, 464 B/s wr, 3 op/s 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[27669]: audit 2024-02-18T01:29:15.032914+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[27669]: audit 2024-02-18T01:29:15.033183+0000 mon.a (mon.0) 2313 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[27669]: audit 2024-02-18T01:29:15.034991+0000 mon.b (mon.1) 338 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[27669]: audit 2024-02-18T01:29:15.035183+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[19734]: audit 2024-02-18T01:29:15.032914+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[19734]: audit 2024-02-18T01:29:15.033183+0000 mon.a (mon.0) 2313 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[19734]: audit 2024-02-18T01:29:15.034991+0000 mon.b (mon.1) 338 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:15 smithi105 bash[19734]: audit 2024-02-18T01:29:15.035183+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:15 smithi155 bash[20805]: audit 2024-02-18T01:29:15.032914+0000 mon.b (mon.1) 337 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:15 smithi155 bash[20805]: audit 2024-02-18T01:29:15.033183+0000 mon.a (mon.0) 2313 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:15 smithi155 bash[20805]: audit 2024-02-18T01:29:15.034991+0000 mon.b (mon.1) 338 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:15 smithi155 bash[20805]: audit 2024-02-18T01:29:15.035183+0000 mon.a (mon.0) 2314 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-63"}]: dispatch 2024-02-18T01:29:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:16 smithi105 bash[19734]: cluster 2024-02-18T01:29:15.807451+0000 mgr.y (mgr.24475) 485 : cluster [DBG] pgmap v686: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-02-18T01:29:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:16 smithi105 bash[19734]: cluster 2024-02-18T01:29:15.881965+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-18T01:29:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:16 smithi105 bash[27669]: cluster 2024-02-18T01:29:15.807451+0000 mgr.y (mgr.24475) 485 : cluster [DBG] pgmap v686: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-02-18T01:29:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:16 smithi105 bash[27669]: cluster 2024-02-18T01:29:15.881965+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-18T01:29:16.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:16] "GET /metrics HTTP/1.1" 200 188457 "" "Prometheus/2.33.4" 2024-02-18T01:29:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:16 smithi155 bash[20805]: cluster 2024-02-18T01:29:15.807451+0000 mgr.y (mgr.24475) 485 : cluster [DBG] pgmap v686: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 409 B/s wr, 3 op/s 2024-02-18T01:29:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:16 smithi155 bash[20805]: cluster 2024-02-18T01:29:15.881965+0000 mon.a (mon.0) 2315 : cluster [DBG] osdmap e395: 8 total, 8 up, 8 in 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[27669]: cluster 2024-02-18T01:29:16.901815+0000 mon.a (mon.0) 2316 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[27669]: audit 2024-02-18T01:29:16.904083+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[27669]: audit 2024-02-18T01:29:16.904901+0000 mon.a (mon.0) 2317 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[19734]: cluster 2024-02-18T01:29:16.901815+0000 mon.a (mon.0) 2316 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[19734]: audit 2024-02-18T01:29:16.904083+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:17 smithi105 bash[19734]: audit 2024-02-18T01:29:16.904901+0000 mon.a (mon.0) 2317 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:18.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:17 smithi155 bash[20805]: cluster 2024-02-18T01:29:16.901815+0000 mon.a (mon.0) 2316 : cluster [DBG] osdmap e396: 8 total, 8 up, 8 in 2024-02-18T01:29:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:17 smithi155 bash[20805]: audit 2024-02-18T01:29:16.904083+0000 mon.b (mon.1) 339 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:18.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:17 smithi155 bash[20805]: audit 2024-02-18T01:29:16.904901+0000 mon.a (mon.0) 2317 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: cluster 2024-02-18T01:29:17.809566+0000 mgr.y (mgr.24475) 486 : cluster [DBG] pgmap v689: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: audit 2024-02-18T01:29:17.901601+0000 mgr.y (mgr.24475) 487 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: audit 2024-02-18T01:29:17.906062+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: cluster 2024-02-18T01:29:17.906429+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: audit 2024-02-18T01:29:17.915340+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[27669]: audit 2024-02-18T01:29:18.874836+0000 mon.a (mon.0) 2320 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: cluster 2024-02-18T01:29:17.809566+0000 mgr.y (mgr.24475) 486 : cluster [DBG] pgmap v689: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: audit 2024-02-18T01:29:17.901601+0000 mgr.y (mgr.24475) 487 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: audit 2024-02-18T01:29:17.906062+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: cluster 2024-02-18T01:29:17.906429+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-18T01:29:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: audit 2024-02-18T01:29:17.915340+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:19.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:18 smithi105 bash[19734]: audit 2024-02-18T01:29:18.874836+0000 mon.a (mon.0) 2320 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: cluster 2024-02-18T01:29:17.809566+0000 mgr.y (mgr.24475) 486 : cluster [DBG] pgmap v689: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: audit 2024-02-18T01:29:17.901601+0000 mgr.y (mgr.24475) 487 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: audit 2024-02-18T01:29:17.906062+0000 mon.a (mon.0) 2318 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: cluster 2024-02-18T01:29:17.906429+0000 mon.a (mon.0) 2319 : cluster [DBG] osdmap e397: 8 total, 8 up, 8 in 2024-02-18T01:29:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: audit 2024-02-18T01:29:17.915340+0000 mon.b (mon.1) 340 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:19.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:18 smithi155 bash[20805]: audit 2024-02-18T01:29:18.874836+0000 mon.a (mon.0) 2320 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: cluster 2024-02-18T01:29:18.927091+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:18.961734+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:18.961759+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:18.962990+0000 mon.a (mon.0) 2323 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:18.963068+0000 mon.b (mon.1) 342 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:19.174177+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:29:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:19.175919+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[27669]: audit 2024-02-18T01:29:19.184033+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: cluster 2024-02-18T01:29:18.927091+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:18.961734+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:18.961759+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:18.962990+0000 mon.a (mon.0) 2323 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:18.963068+0000 mon.b (mon.1) 342 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:19.174177+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:19.175919+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:29:20.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:19 smithi105 bash[19734]: audit 2024-02-18T01:29:19.184033+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: cluster 2024-02-18T01:29:18.927091+0000 mon.a (mon.0) 2321 : cluster [DBG] osdmap e398: 8 total, 8 up, 8 in 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:18.961734+0000 mon.b (mon.1) 341 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:18.961759+0000 mon.a (mon.0) 2322 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:18.962990+0000 mon.a (mon.0) 2323 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:18.963068+0000 mon.b (mon.1) 342 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-65"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:19.174177+0000 mon.a (mon.0) 2324 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:19.175919+0000 mon.a (mon.0) 2325 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:29:20.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:19 smithi155 bash[20805]: audit 2024-02-18T01:29:19.184033+0000 mon.a (mon.0) 2326 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:29:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:20 smithi105 bash[19734]: cluster 2024-02-18T01:29:19.810509+0000 mgr.y (mgr.24475) 488 : cluster [DBG] pgmap v692: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:20 smithi105 bash[19734]: cluster 2024-02-18T01:29:19.948346+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-18T01:29:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:20 smithi105 bash[27669]: cluster 2024-02-18T01:29:19.810509+0000 mgr.y (mgr.24475) 488 : cluster [DBG] pgmap v692: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:20 smithi105 bash[27669]: cluster 2024-02-18T01:29:19.948346+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-18T01:29:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:20 smithi155 bash[20805]: cluster 2024-02-18T01:29:19.810509+0000 mgr.y (mgr.24475) 488 : cluster [DBG] pgmap v692: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:20 smithi155 bash[20805]: cluster 2024-02-18T01:29:19.948346+0000 mon.a (mon.0) 2327 : cluster [DBG] osdmap e399: 8 total, 8 up, 8 in 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[19734]: cluster 2024-02-18T01:29:20.961232+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[19734]: audit 2024-02-18T01:29:20.964216+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[19734]: audit 2024-02-18T01:29:20.965956+0000 mon.a (mon.0) 2329 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[27669]: cluster 2024-02-18T01:29:20.961232+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[27669]: audit 2024-02-18T01:29:20.964216+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:21 smithi105 bash[27669]: audit 2024-02-18T01:29:20.965956+0000 mon.a (mon.0) 2329 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:21 smithi155 bash[20805]: cluster 2024-02-18T01:29:20.961232+0000 mon.a (mon.0) 2328 : cluster [DBG] osdmap e400: 8 total, 8 up, 8 in 2024-02-18T01:29:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:21 smithi155 bash[20805]: audit 2024-02-18T01:29:20.964216+0000 mon.b (mon.1) 343 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:22.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:21 smithi155 bash[20805]: audit 2024-02-18T01:29:20.965956+0000 mon.a (mon.0) 2329 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: cluster 2024-02-18T01:29:21.811334+0000 mgr.y (mgr.24475) 489 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:21.964812+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: cluster 2024-02-18T01:29:21.965006+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:21.969147+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:22.014346+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:22.014347+0000 mon.a (mon.0) 2332 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:22.015402+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:23.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[27669]: audit 2024-02-18T01:29:22.015515+0000 mon.b (mon.1) 346 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: cluster 2024-02-18T01:29:21.811334+0000 mgr.y (mgr.24475) 489 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:21.964812+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: cluster 2024-02-18T01:29:21.965006+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:21.969147+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:22.014346+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:22.014347+0000 mon.a (mon.0) 2332 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:22.015402+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:23.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:23 smithi105 bash[19734]: audit 2024-02-18T01:29:22.015515+0000 mon.b (mon.1) 346 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: cluster 2024-02-18T01:29:21.811334+0000 mgr.y (mgr.24475) 489 : cluster [DBG] pgmap v695: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:21.964812+0000 mon.a (mon.0) 2330 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: cluster 2024-02-18T01:29:21.965006+0000 mon.a (mon.0) 2331 : cluster [DBG] osdmap e401: 8 total, 8 up, 8 in 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:21.969147+0000 mon.b (mon.1) 344 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:22.014346+0000 mon.b (mon.1) 345 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:22.014347+0000 mon.a (mon.0) 2332 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:22.015402+0000 mon.a (mon.0) 2333 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:23.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:23 smithi155 bash[20805]: audit 2024-02-18T01:29:22.015515+0000 mon.b (mon.1) 346 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-67"}]: dispatch 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[19734]: cluster 2024-02-18T01:29:22.984558+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[19734]: cluster 2024-02-18T01:29:23.812127+0000 mgr.y (mgr.24475) 490 : cluster [DBG] pgmap v698: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[19734]: cluster 2024-02-18T01:29:23.987332+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[19734]: audit 2024-02-18T01:29:23.990049+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[19734]: audit 2024-02-18T01:29:23.992366+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:24.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[27669]: cluster 2024-02-18T01:29:22.984558+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-18T01:29:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[27669]: cluster 2024-02-18T01:29:23.812127+0000 mgr.y (mgr.24475) 490 : cluster [DBG] pgmap v698: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[27669]: cluster 2024-02-18T01:29:23.987332+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-18T01:29:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[27669]: audit 2024-02-18T01:29:23.990049+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:24 smithi105 bash[27669]: audit 2024-02-18T01:29:23.992366+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:24 smithi155 bash[20805]: cluster 2024-02-18T01:29:22.984558+0000 mon.a (mon.0) 2334 : cluster [DBG] osdmap e402: 8 total, 8 up, 8 in 2024-02-18T01:29:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:24 smithi155 bash[20805]: cluster 2024-02-18T01:29:23.812127+0000 mgr.y (mgr.24475) 490 : cluster [DBG] pgmap v698: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:24 smithi155 bash[20805]: cluster 2024-02-18T01:29:23.987332+0000 mon.a (mon.0) 2335 : cluster [DBG] osdmap e403: 8 total, 8 up, 8 in 2024-02-18T01:29:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:24 smithi155 bash[20805]: audit 2024-02-18T01:29:23.990049+0000 mon.b (mon.1) 347 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:24 smithi155 bash[20805]: audit 2024-02-18T01:29:23.992366+0000 mon.a (mon.0) 2336 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[19734]: audit 2024-02-18T01:29:24.992734+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[19734]: cluster 2024-02-18T01:29:24.993279+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[19734]: audit 2024-02-18T01:29:24.995728+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[19734]: audit 2024-02-18T01:29:25.006542+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[19734]: audit 2024-02-18T01:29:25.006930+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[27669]: audit 2024-02-18T01:29:24.992734+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[27669]: cluster 2024-02-18T01:29:24.993279+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-18T01:29:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[27669]: audit 2024-02-18T01:29:24.995728+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[27669]: audit 2024-02-18T01:29:25.006542+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:25 smithi105 bash[27669]: audit 2024-02-18T01:29:25.006930+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:25 smithi155 bash[20805]: audit 2024-02-18T01:29:24.992734+0000 mon.a (mon.0) 2337 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:25 smithi155 bash[20805]: cluster 2024-02-18T01:29:24.993279+0000 mon.a (mon.0) 2338 : cluster [DBG] osdmap e404: 8 total, 8 up, 8 in 2024-02-18T01:29:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:25 smithi155 bash[20805]: audit 2024-02-18T01:29:24.995728+0000 mon.b (mon.1) 348 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:25 smithi155 bash[20805]: audit 2024-02-18T01:29:25.006542+0000 mon.b (mon.1) 349 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:25 smithi155 bash[20805]: audit 2024-02-18T01:29:25.006930+0000 mon.a (mon.0) 2339 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:29:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:26] "GET /metrics HTTP/1.1" 200 188548 "" "Prometheus/2.33.4" 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[19734]: cluster 2024-02-18T01:29:25.812925+0000 mgr.y (mgr.24475) 491 : cluster [DBG] pgmap v701: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[19734]: audit 2024-02-18T01:29:26.000650+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[19734]: cluster 2024-02-18T01:29:26.000865+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[27669]: cluster 2024-02-18T01:29:25.812925+0000 mgr.y (mgr.24475) 491 : cluster [DBG] pgmap v701: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[27669]: audit 2024-02-18T01:29:26.000650+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:27.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:26 smithi105 bash[27669]: cluster 2024-02-18T01:29:26.000865+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-18T01:29:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:26 smithi155 bash[20805]: cluster 2024-02-18T01:29:25.812925+0000 mgr.y (mgr.24475) 491 : cluster [DBG] pgmap v701: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:29:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:26 smithi155 bash[20805]: audit 2024-02-18T01:29:26.000650+0000 mon.a (mon.0) 2340 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:29:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:26 smithi155 bash[20805]: cluster 2024-02-18T01:29:26.000865+0000 mon.a (mon.0) 2341 : cluster [DBG] osdmap e405: 8 total, 8 up, 8 in 2024-02-18T01:29:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:28 smithi155 bash[20805]: cluster 2024-02-18T01:29:27.007768+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-18T01:29:28.244 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:28.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:28 smithi105 bash[19734]: cluster 2024-02-18T01:29:27.007768+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-18T01:29:28.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:28 smithi105 bash[27669]: cluster 2024-02-18T01:29:27.007768+0000 mon.a (mon.0) 2342 : cluster [DBG] osdmap e406: 8 total, 8 up, 8 in 2024-02-18T01:29:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:29 smithi105 bash[27669]: cluster 2024-02-18T01:29:27.815048+0000 mgr.y (mgr.24475) 492 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-02-18T01:29:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:29 smithi105 bash[27669]: audit 2024-02-18T01:29:27.913539+0000 mgr.y (mgr.24475) 493 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:29 smithi105 bash[19734]: cluster 2024-02-18T01:29:27.815048+0000 mgr.y (mgr.24475) 492 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-02-18T01:29:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:29 smithi105 bash[19734]: audit 2024-02-18T01:29:27.913539+0000 mgr.y (mgr.24475) 493 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:29 smithi155 bash[20805]: cluster 2024-02-18T01:29:27.815048+0000 mgr.y (mgr.24475) 492 : cluster [DBG] pgmap v704: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.2 KiB/s wr, 7 op/s 2024-02-18T01:29:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:29 smithi155 bash[20805]: audit 2024-02-18T01:29:27.913539+0000 mgr.y (mgr.24475) 493 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:31.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:31 smithi105 bash[27669]: cluster 2024-02-18T01:29:29.815879+0000 mgr.y (mgr.24475) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T01:29:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:31 smithi105 bash[19734]: cluster 2024-02-18T01:29:29.815879+0000 mgr.y (mgr.24475) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T01:29:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:31 smithi155 bash[20805]: cluster 2024-02-18T01:29:29.815879+0000 mgr.y (mgr.24475) 494 : cluster [DBG] pgmap v705: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.2 KiB/s wr, 5 op/s 2024-02-18T01:29:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:33.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:33 smithi105 bash[19734]: cluster 2024-02-18T01:29:31.816674+0000 mgr.y (mgr.24475) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:29:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:33 smithi105 bash[27669]: cluster 2024-02-18T01:29:31.816674+0000 mgr.y (mgr.24475) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:29:33.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:33 smithi155 bash[20805]: cluster 2024-02-18T01:29:31.816674+0000 mgr.y (mgr.24475) 495 : cluster [DBG] pgmap v706: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 1.9 KiB/s wr, 4 op/s 2024-02-18T01:29:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:35 smithi105 bash[27669]: cluster 2024-02-18T01:29:33.818984+0000 mgr.y (mgr.24475) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-18T01:29:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:35 smithi105 bash[19734]: cluster 2024-02-18T01:29:33.818984+0000 mgr.y (mgr.24475) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-18T01:29:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:35 smithi155 bash[20805]: cluster 2024-02-18T01:29:33.818984+0000 mgr.y (mgr.24475) 496 : cluster [DBG] pgmap v707: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.6 KiB/s wr, 4 op/s 2024-02-18T01:29:36.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:36] "GET /metrics HTTP/1.1" 200 188714 "" "Prometheus/2.33.4" 2024-02-18T01:29:37.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:37 smithi105 bash[19734]: cluster 2024-02-18T01:29:35.819941+0000 mgr.y (mgr.24475) 497 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-18T01:29:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:37 smithi105 bash[27669]: cluster 2024-02-18T01:29:35.819941+0000 mgr.y (mgr.24475) 497 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-18T01:29:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:37 smithi155 bash[20805]: cluster 2024-02-18T01:29:35.819941+0000 mgr.y (mgr.24475) 497 : cluster [DBG] pgmap v708: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-02-18T01:29:38.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:39 smithi105 bash[19734]: cluster 2024-02-18T01:29:37.822162+0000 mgr.y (mgr.24475) 498 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:39 smithi105 bash[19734]: audit 2024-02-18T01:29:37.924843+0000 mgr.y (mgr.24475) 499 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:39 smithi105 bash[27669]: cluster 2024-02-18T01:29:37.822162+0000 mgr.y (mgr.24475) 498 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:39 smithi105 bash[27669]: audit 2024-02-18T01:29:37.924843+0000 mgr.y (mgr.24475) 499 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:39 smithi155 bash[20805]: cluster 2024-02-18T01:29:37.822162+0000 mgr.y (mgr.24475) 498 : cluster [DBG] pgmap v709: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:39 smithi155 bash[20805]: audit 2024-02-18T01:29:37.924843+0000 mgr.y (mgr.24475) 499 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:41.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:41 smithi105 bash[19734]: cluster 2024-02-18T01:29:39.822997+0000 mgr.y (mgr.24475) 500 : cluster [DBG] pgmap v710: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:41 smithi105 bash[27669]: cluster 2024-02-18T01:29:39.822997+0000 mgr.y (mgr.24475) 500 : cluster [DBG] pgmap v710: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:41 smithi155 bash[20805]: cluster 2024-02-18T01:29:39.822997+0000 mgr.y (mgr.24475) 500 : cluster [DBG] pgmap v710: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:43 smithi105 bash[19734]: cluster 2024-02-18T01:29:41.823806+0000 mgr.y (mgr.24475) 501 : cluster [DBG] pgmap v711: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:43.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:43 smithi105 bash[27669]: cluster 2024-02-18T01:29:41.823806+0000 mgr.y (mgr.24475) 501 : cluster [DBG] pgmap v711: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:43 smithi155 bash[20805]: cluster 2024-02-18T01:29:41.823806+0000 mgr.y (mgr.24475) 501 : cluster [DBG] pgmap v711: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:45 smithi105 bash[19734]: cluster 2024-02-18T01:29:43.826021+0000 mgr.y (mgr.24475) 502 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:45 smithi105 bash[27669]: cluster 2024-02-18T01:29:43.826021+0000 mgr.y (mgr.24475) 502 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:45 smithi155 bash[20805]: cluster 2024-02-18T01:29:43.826021+0000 mgr.y (mgr.24475) 502 : cluster [DBG] pgmap v712: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:46 smithi105 bash[19734]: cluster 2024-02-18T01:29:45.826994+0000 mgr.y (mgr.24475) 503 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:46 smithi105 bash[27669]: cluster 2024-02-18T01:29:45.826994+0000 mgr.y (mgr.24475) 503 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:46 smithi155 bash[20805]: cluster 2024-02-18T01:29:45.826994+0000 mgr.y (mgr.24475) 503 : cluster [DBG] pgmap v713: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:46] "GET /metrics HTTP/1.1" 200 188714 "" "Prometheus/2.33.4" 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[19734]: audit 2024-02-18T01:29:47.085402+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[19734]: audit 2024-02-18T01:29:47.085640+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[19734]: audit 2024-02-18T01:29:47.087692+0000 mon.b (mon.1) 351 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[19734]: audit 2024-02-18T01:29:47.087749+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[27669]: audit 2024-02-18T01:29:47.085402+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[27669]: audit 2024-02-18T01:29:47.085640+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[27669]: audit 2024-02-18T01:29:47.087692+0000 mon.b (mon.1) 351 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:47.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:47 smithi105 bash[27669]: audit 2024-02-18T01:29:47.087749+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:47 smithi155 bash[20805]: audit 2024-02-18T01:29:47.085402+0000 mon.b (mon.1) 350 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:47 smithi155 bash[20805]: audit 2024-02-18T01:29:47.085640+0000 mon.a (mon.0) 2343 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:29:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:47 smithi155 bash[20805]: audit 2024-02-18T01:29:47.087692+0000 mon.b (mon.1) 351 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:47 smithi155 bash[20805]: audit 2024-02-18T01:29:47.087749+0000 mon.a (mon.0) 2344 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-69"}]: dispatch 2024-02-18T01:29:48.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:48 smithi105 bash[27669]: cluster 2024-02-18T01:29:47.829174+0000 mgr.y (mgr.24475) 504 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:48 smithi105 bash[27669]: audit 2024-02-18T01:29:47.936104+0000 mgr.y (mgr.24475) 505 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:48 smithi105 bash[19734]: cluster 2024-02-18T01:29:47.829174+0000 mgr.y (mgr.24475) 504 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:48 smithi105 bash[19734]: audit 2024-02-18T01:29:47.936104+0000 mgr.y (mgr.24475) 505 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:48 smithi155 bash[20805]: cluster 2024-02-18T01:29:47.829174+0000 mgr.y (mgr.24475) 504 : cluster [DBG] pgmap v714: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:29:48.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:48 smithi155 bash[20805]: audit 2024-02-18T01:29:47.936104+0000 mgr.y (mgr.24475) 505 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[27669]: cluster 2024-02-18T01:29:48.151947+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[27669]: cluster 2024-02-18T01:29:49.153541+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[27669]: audit 2024-02-18T01:29:49.157581+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[27669]: audit 2024-02-18T01:29:49.160288+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[19734]: cluster 2024-02-18T01:29:48.151947+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[19734]: cluster 2024-02-18T01:29:49.153541+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-18T01:29:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[19734]: audit 2024-02-18T01:29:49.157581+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:49.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:49 smithi105 bash[19734]: audit 2024-02-18T01:29:49.160288+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:49 smithi155 bash[20805]: cluster 2024-02-18T01:29:48.151947+0000 mon.a (mon.0) 2345 : cluster [DBG] osdmap e407: 8 total, 8 up, 8 in 2024-02-18T01:29:49.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:49 smithi155 bash[20805]: cluster 2024-02-18T01:29:49.153541+0000 mon.a (mon.0) 2346 : cluster [DBG] osdmap e408: 8 total, 8 up, 8 in 2024-02-18T01:29:49.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:49 smithi155 bash[20805]: audit 2024-02-18T01:29:49.157581+0000 mon.b (mon.1) 352 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:49.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:49 smithi155 bash[20805]: audit 2024-02-18T01:29:49.160288+0000 mon.a (mon.0) 2347 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:29:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[19734]: cluster 2024-02-18T01:29:49.829993+0000 mgr.y (mgr.24475) 506 : cluster [DBG] pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[19734]: audit 2024-02-18T01:29:50.156878+0000 mon.a (mon.0) 2348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[19734]: cluster 2024-02-18T01:29:50.157697+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-18T01:29:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[19734]: audit 2024-02-18T01:29:50.159892+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:50.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[27669]: cluster 2024-02-18T01:29:49.829993+0000 mgr.y (mgr.24475) 506 : cluster [DBG] pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:50.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[27669]: audit 2024-02-18T01:29:50.156878+0000 mon.a (mon.0) 2348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:50.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[27669]: cluster 2024-02-18T01:29:50.157697+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-18T01:29:50.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:50 smithi105 bash[27669]: audit 2024-02-18T01:29:50.159892+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:50 smithi155 bash[20805]: cluster 2024-02-18T01:29:49.829993+0000 mgr.y (mgr.24475) 506 : cluster [DBG] pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:50 smithi155 bash[20805]: audit 2024-02-18T01:29:50.156878+0000 mon.a (mon.0) 2348 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:29:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:50 smithi155 bash[20805]: cluster 2024-02-18T01:29:50.157697+0000 mon.a (mon.0) 2349 : cluster [DBG] osdmap e409: 8 total, 8 up, 8 in 2024-02-18T01:29:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:50 smithi155 bash[20805]: audit 2024-02-18T01:29:50.159892+0000 mon.b (mon.1) 353 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:29:52.167 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:29:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:29:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:52 smithi155 bash[20805]: cluster 2024-02-18T01:29:51.157055+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-18T01:29:52.168 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:52 smithi155 bash[20805]: cluster 2024-02-18T01:29:51.830832+0000 mgr.y (mgr.24475) 507 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:52 smithi105 bash[19734]: cluster 2024-02-18T01:29:51.157055+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-18T01:29:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:52 smithi105 bash[19734]: cluster 2024-02-18T01:29:51.830832+0000 mgr.y (mgr.24475) 507 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:52 smithi105 bash[27669]: cluster 2024-02-18T01:29:51.157055+0000 mon.a (mon.0) 2350 : cluster [DBG] osdmap e410: 8 total, 8 up, 8 in 2024-02-18T01:29:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:52 smithi105 bash[27669]: cluster 2024-02-18T01:29:51.830832+0000 mgr.y (mgr.24475) 507 : cluster [DBG] pgmap v720: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:29:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:53 smithi105 bash[19734]: cluster 2024-02-18T01:29:52.161044+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-18T01:29:53.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:53 smithi105 bash[27669]: cluster 2024-02-18T01:29:52.161044+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-18T01:29:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:53 smithi155 bash[20805]: cluster 2024-02-18T01:29:52.161044+0000 mon.a (mon.0) 2351 : cluster [DBG] osdmap e411: 8 total, 8 up, 8 in 2024-02-18T01:29:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:54 smithi105 bash[27669]: cluster 2024-02-18T01:29:53.832951+0000 mgr.y (mgr.24475) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:54.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:54 smithi105 bash[19734]: cluster 2024-02-18T01:29:53.832951+0000 mgr.y (mgr.24475) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:54 smithi155 bash[20805]: cluster 2024-02-18T01:29:53.832951+0000 mgr.y (mgr.24475) 508 : cluster [DBG] pgmap v722: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-02-18T01:29:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:56 smithi105 bash[27669]: cluster 2024-02-18T01:29:55.833804+0000 mgr.y (mgr.24475) 509 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-18T01:29:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:56 smithi105 bash[19734]: cluster 2024-02-18T01:29:55.833804+0000 mgr.y (mgr.24475) 509 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-18T01:29:56.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:29:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:29:56] "GET /metrics HTTP/1.1" 200 188796 "" "Prometheus/2.33.4" 2024-02-18T01:29:57.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:56 smithi155 bash[20805]: cluster 2024-02-18T01:29:55.833804+0000 mgr.y (mgr.24475) 509 : cluster [DBG] pgmap v723: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-02-18T01:29:58.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:29:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:29:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:58 smithi105 bash[19734]: cluster 2024-02-18T01:29:57.835928+0000 mgr.y (mgr.24475) 510 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 932 B/s wr, 4 op/s 2024-02-18T01:29:59.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:29:58 smithi105 bash[19734]: audit 2024-02-18T01:29:57.945538+0000 mgr.y (mgr.24475) 511 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:58 smithi105 bash[27669]: cluster 2024-02-18T01:29:57.835928+0000 mgr.y (mgr.24475) 510 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 932 B/s wr, 4 op/s 2024-02-18T01:29:59.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:29:58 smithi105 bash[27669]: audit 2024-02-18T01:29:57.945538+0000 mgr.y (mgr.24475) 511 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:29:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:58 smithi155 bash[20805]: cluster 2024-02-18T01:29:57.835928+0000 mgr.y (mgr.24475) 510 : cluster [DBG] pgmap v724: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 932 B/s wr, 4 op/s 2024-02-18T01:29:59.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:29:58 smithi155 bash[20805]: audit 2024-02-18T01:29:57.945538+0000 mgr.y (mgr.24475) 511 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:01.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[19734]: cluster 2024-02-18T01:29:59.836768+0000 mgr.y (mgr.24475) 512 : cluster [DBG] pgmap v725: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 825 B/s wr, 3 op/s 2024-02-18T01:30:01.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:00.000137+0000 mon.a (mon.0) 2352 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.243 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:00.000183+0000 mon.a (mon.0) 2353 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:00.000201+0000 mon.a (mon.0) 2354 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:00.000216+0000 mon.a (mon.0) 2355 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[27669]: cluster 2024-02-18T01:29:59.836768+0000 mgr.y (mgr.24475) 512 : cluster [DBG] pgmap v725: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 825 B/s wr, 3 op/s 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:00.000137+0000 mon.a (mon.0) 2352 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:00.000183+0000 mon.a (mon.0) 2353 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:00.000201+0000 mon.a (mon.0) 2354 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T01:30:01.244 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:00.000216+0000 mon.a (mon.0) 2355 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T01:30:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:00 smithi155 bash[20805]: cluster 2024-02-18T01:29:59.836768+0000 mgr.y (mgr.24475) 512 : cluster [DBG] pgmap v725: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 825 B/s wr, 3 op/s 2024-02-18T01:30:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:00.000137+0000 mon.a (mon.0) 2352 : cluster [WRN] Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:00.000183+0000 mon.a (mon.0) 2353 : cluster [WRN] [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-02-18T01:30:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:00.000201+0000 mon.a (mon.0) 2354 : cluster [WRN] application not enabled on pool 'ceph_test_rados_api_asio' 2024-02-18T01:30:01.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:00.000216+0000 mon.a (mon.0) 2355 : cluster [WRN] use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-02-18T01:30:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[27669]: cluster 2024-02-18T01:30:01.837629+0000 mgr.y (mgr.24475) 513 : cluster [DBG] pgmap v726: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[27669]: audit 2024-02-18T01:30:02.225456+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[27669]: audit 2024-02-18T01:30:02.225630+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[27669]: audit 2024-02-18T01:30:02.227707+0000 mon.b (mon.1) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[27669]: audit 2024-02-18T01:30:02.227723+0000 mon.a (mon.0) 2357 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[19734]: cluster 2024-02-18T01:30:01.837629+0000 mgr.y (mgr.24475) 513 : cluster [DBG] pgmap v726: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-18T01:30:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[19734]: audit 2024-02-18T01:30:02.225456+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[19734]: audit 2024-02-18T01:30:02.225630+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[19734]: audit 2024-02-18T01:30:02.227707+0000 mon.b (mon.1) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:03.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:02 smithi105 bash[19734]: audit 2024-02-18T01:30:02.227723+0000 mon.a (mon.0) 2357 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:02 smithi155 bash[20805]: cluster 2024-02-18T01:30:01.837629+0000 mgr.y (mgr.24475) 513 : cluster [DBG] pgmap v726: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 716 B/s wr, 3 op/s 2024-02-18T01:30:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:02 smithi155 bash[20805]: audit 2024-02-18T01:30:02.225456+0000 mon.b (mon.1) 354 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:02 smithi155 bash[20805]: audit 2024-02-18T01:30:02.225630+0000 mon.a (mon.0) 2356 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:02 smithi155 bash[20805]: audit 2024-02-18T01:30:02.227707+0000 mon.b (mon.1) 355 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:02 smithi155 bash[20805]: audit 2024-02-18T01:30:02.227723+0000 mon.a (mon.0) 2357 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-71"}]: dispatch 2024-02-18T01:30:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:03 smithi105 bash[19734]: cluster 2024-02-18T01:30:02.911467+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-18T01:30:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:03 smithi105 bash[27669]: cluster 2024-02-18T01:30:02.911467+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-18T01:30:04.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:03 smithi155 bash[20805]: cluster 2024-02-18T01:30:02.911467+0000 mon.a (mon.0) 2358 : cluster [DBG] osdmap e412: 8 total, 8 up, 8 in 2024-02-18T01:30:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[19734]: cluster 2024-02-18T01:30:03.838460+0000 mgr.y (mgr.24475) 514 : cluster [DBG] pgmap v728: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:30:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[19734]: cluster 2024-02-18T01:30:03.933434+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-18T01:30:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[19734]: audit 2024-02-18T01:30:03.939622+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[19734]: audit 2024-02-18T01:30:03.943368+0000 mon.a (mon.0) 2360 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[27669]: cluster 2024-02-18T01:30:03.838460+0000 mgr.y (mgr.24475) 514 : cluster [DBG] pgmap v728: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:30:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[27669]: cluster 2024-02-18T01:30:03.933434+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-18T01:30:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[27669]: audit 2024-02-18T01:30:03.939622+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:05.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:04 smithi105 bash[27669]: audit 2024-02-18T01:30:03.943368+0000 mon.a (mon.0) 2360 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:04 smithi155 bash[20805]: cluster 2024-02-18T01:30:03.838460+0000 mgr.y (mgr.24475) 514 : cluster [DBG] pgmap v728: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:30:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:04 smithi155 bash[20805]: cluster 2024-02-18T01:30:03.933434+0000 mon.a (mon.0) 2359 : cluster [DBG] osdmap e413: 8 total, 8 up, 8 in 2024-02-18T01:30:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:04 smithi155 bash[20805]: audit 2024-02-18T01:30:03.939622+0000 mon.b (mon.1) 356 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:05.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:04 smithi155 bash[20805]: audit 2024-02-18T01:30:03.943368+0000 mon.a (mon.0) 2360 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[27669]: audit 2024-02-18T01:30:04.938767+0000 mon.a (mon.0) 2361 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:06.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[27669]: cluster 2024-02-18T01:30:04.939009+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-18T01:30:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[27669]: audit 2024-02-18T01:30:04.941034+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:06.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[27669]: cluster 2024-02-18T01:30:05.938936+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-18T01:30:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[19734]: audit 2024-02-18T01:30:04.938767+0000 mon.a (mon.0) 2361 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[19734]: cluster 2024-02-18T01:30:04.939009+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-18T01:30:06.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[19734]: audit 2024-02-18T01:30:04.941034+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:06.242 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:05 smithi105 bash[19734]: cluster 2024-02-18T01:30:05.938936+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-18T01:30:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:05 smithi155 bash[20805]: audit 2024-02-18T01:30:04.938767+0000 mon.a (mon.0) 2361 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:05 smithi155 bash[20805]: cluster 2024-02-18T01:30:04.939009+0000 mon.a (mon.0) 2362 : cluster [DBG] osdmap e414: 8 total, 8 up, 8 in 2024-02-18T01:30:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:05 smithi155 bash[20805]: audit 2024-02-18T01:30:04.941034+0000 mon.b (mon.1) 357 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:06.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:05 smithi155 bash[20805]: cluster 2024-02-18T01:30:05.938936+0000 mon.a (mon.0) 2363 : cluster [DBG] osdmap e415: 8 total, 8 up, 8 in 2024-02-18T01:30:06.958 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:06] "GET /metrics HTTP/1.1" 200 188923 "" "Prometheus/2.33.4" 2024-02-18T01:30:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[27669]: cluster 2024-02-18T01:30:05.839278+0000 mgr.y (mgr.24475) 515 : cluster [DBG] pgmap v731: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:07.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[27669]: cluster 2024-02-18T01:30:06.939604+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-18T01:30:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[19734]: cluster 2024-02-18T01:30:05.839278+0000 mgr.y (mgr.24475) 515 : cluster [DBG] pgmap v731: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:07.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[19734]: cluster 2024-02-18T01:30:06.939604+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-18T01:30:07.241 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:30:06 smithi105 bash[36150]: debug 2024-02-18T01:30:06.976+0000 7fcc9bf94700 -1 snap_mapper.add_oid found existing snaps mapped on 63:efeadd2a:test-rados-api-smithi105-61833-74::foo:24, removing 2024-02-18T01:30:07.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:06 smithi155 bash[20805]: cluster 2024-02-18T01:30:05.839278+0000 mgr.y (mgr.24475) 515 : cluster [DBG] pgmap v731: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:07.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:06 smithi155 bash[20805]: cluster 2024-02-18T01:30:06.939604+0000 mon.a (mon.0) 2364 : cluster [DBG] osdmap e416: 8 total, 8 up, 8 in 2024-02-18T01:30:07.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:30:06 smithi155 bash[24809]: debug 2024-02-18T01:30:06.976+0000 7fc8bf652700 -1 snap_mapper.add_oid found existing snaps mapped on 63:efeadd2a:test-rados-api-smithi105-61833-74::foo:24, removing 2024-02-18T01:30:07.244 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:30:06 smithi155 bash[29937]: debug 2024-02-18T01:30:06.976+0000 7f288e528700 -1 snap_mapper.add_oid found existing snaps mapped on 63:efeadd2a:test-rados-api-smithi105-61833-74::foo:24, removing 2024-02-18T01:30:08.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[19734]: audit 2024-02-18T01:30:07.004445+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[19734]: audit 2024-02-18T01:30:07.004531+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[19734]: audit 2024-02-18T01:30:07.005957+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[19734]: audit 2024-02-18T01:30:07.006157+0000 mon.b (mon.1) 359 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[27669]: audit 2024-02-18T01:30:07.004445+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[27669]: audit 2024-02-18T01:30:07.004531+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[27669]: audit 2024-02-18T01:30:07.005957+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:08.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:07 smithi105 bash[27669]: audit 2024-02-18T01:30:07.006157+0000 mon.b (mon.1) 359 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:08.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:07 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:08.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:07 smithi155 bash[20805]: audit 2024-02-18T01:30:07.004445+0000 mon.a (mon.0) 2365 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:07 smithi155 bash[20805]: audit 2024-02-18T01:30:07.004531+0000 mon.b (mon.1) 358 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:08.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:07 smithi155 bash[20805]: audit 2024-02-18T01:30:07.005957+0000 mon.a (mon.0) 2366 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:08.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:07 smithi155 bash[20805]: audit 2024-02-18T01:30:07.006157+0000 mon.b (mon.1) 359 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-73"}]: dispatch 2024-02-18T01:30:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[19734]: cluster 2024-02-18T01:30:07.841444+0000 mgr.y (mgr.24475) 516 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-02-18T01:30:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[19734]: audit 2024-02-18T01:30:07.948235+0000 mgr.y (mgr.24475) 517 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[19734]: cluster 2024-02-18T01:30:07.963985+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-18T01:30:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[27669]: cluster 2024-02-18T01:30:07.841444+0000 mgr.y (mgr.24475) 516 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-02-18T01:30:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[27669]: audit 2024-02-18T01:30:07.948235+0000 mgr.y (mgr.24475) 517 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:09.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:08 smithi105 bash[27669]: cluster 2024-02-18T01:30:07.963985+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-18T01:30:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:08 smithi155 bash[20805]: cluster 2024-02-18T01:30:07.841444+0000 mgr.y (mgr.24475) 516 : cluster [DBG] pgmap v734: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 3.0 KiB/s wr, 10 op/s 2024-02-18T01:30:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:08 smithi155 bash[20805]: audit 2024-02-18T01:30:07.948235+0000 mgr.y (mgr.24475) 517 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:08 smithi155 bash[20805]: cluster 2024-02-18T01:30:07.963985+0000 mon.a (mon.0) 2367 : cluster [DBG] osdmap e417: 8 total, 8 up, 8 in 2024-02-18T01:30:10.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:09 smithi155 bash[20805]: cluster 2024-02-18T01:30:08.992712+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-18T01:30:10.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:09 smithi155 bash[20805]: audit 2024-02-18T01:30:08.997895+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:10.245 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:09 smithi155 bash[20805]: audit 2024-02-18T01:30:08.998777+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[19734]: cluster 2024-02-18T01:30:08.992712+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[19734]: audit 2024-02-18T01:30:08.997895+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[19734]: audit 2024-02-18T01:30:08.998777+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[27669]: cluster 2024-02-18T01:30:08.992712+0000 mon.a (mon.0) 2368 : cluster [DBG] osdmap e418: 8 total, 8 up, 8 in 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[27669]: audit 2024-02-18T01:30:08.997895+0000 mon.b (mon.1) 360 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:09 smithi105 bash[27669]: audit 2024-02-18T01:30:08.998777+0000 mon.a (mon.0) 2369 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[19734]: cluster 2024-02-18T01:30:09.842288+0000 mgr.y (mgr.24475) 518 : cluster [DBG] pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[19734]: audit 2024-02-18T01:30:10.006261+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[19734]: cluster 2024-02-18T01:30:10.006538+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[19734]: audit 2024-02-18T01:30:10.008966+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[27669]: cluster 2024-02-18T01:30:09.842288+0000 mgr.y (mgr.24475) 518 : cluster [DBG] pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T01:30:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[27669]: audit 2024-02-18T01:30:10.006261+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:11.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[27669]: cluster 2024-02-18T01:30:10.006538+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-18T01:30:11.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:11 smithi105 bash[27669]: audit 2024-02-18T01:30:10.008966+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:11 smithi155 bash[20805]: cluster 2024-02-18T01:30:09.842288+0000 mgr.y (mgr.24475) 518 : cluster [DBG] pgmap v737: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 3 op/s 2024-02-18T01:30:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:11 smithi155 bash[20805]: audit 2024-02-18T01:30:10.006261+0000 mon.a (mon.0) 2370 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:11 smithi155 bash[20805]: cluster 2024-02-18T01:30:10.006538+0000 mon.a (mon.0) 2371 : cluster [DBG] osdmap e419: 8 total, 8 up, 8 in 2024-02-18T01:30:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:11 smithi155 bash[20805]: audit 2024-02-18T01:30:10.008966+0000 mon.b (mon.1) 361 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:12 smithi155 bash[20805]: cluster 2024-02-18T01:30:11.015171+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-18T01:30:12.244 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:12 smithi105 bash[19734]: cluster 2024-02-18T01:30:11.015171+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-18T01:30:12.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:12 smithi105 bash[27669]: cluster 2024-02-18T01:30:11.015171+0000 mon.a (mon.0) 2372 : cluster [DBG] osdmap e420: 8 total, 8 up, 8 in 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: cluster 2024-02-18T01:30:11.843165+0000 mgr.y (mgr.24475) 519 : cluster [DBG] pgmap v740: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: cluster 2024-02-18T01:30:12.018974+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: audit 2024-02-18T01:30:12.035426+0000 mgr.y (mgr.24475) 520 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: audit 2024-02-18T01:30:12.035843+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: cluster 2024-02-18T01:30:12.150321+0000 osd.1 (osd.1) 13 : cluster [DBG] 219.15 deep-scrub starts 2024-02-18T01:30:13.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[27669]: cluster 2024-02-18T01:30:12.155558+0000 osd.1 (osd.1) 14 : cluster [DBG] 219.15 deep-scrub ok 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: cluster 2024-02-18T01:30:11.843165+0000 mgr.y (mgr.24475) 519 : cluster [DBG] pgmap v740: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: cluster 2024-02-18T01:30:12.018974+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: audit 2024-02-18T01:30:12.035426+0000 mgr.y (mgr.24475) 520 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: audit 2024-02-18T01:30:12.035843+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: cluster 2024-02-18T01:30:12.150321+0000 osd.1 (osd.1) 13 : cluster [DBG] 219.15 deep-scrub starts 2024-02-18T01:30:13.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:13 smithi105 bash[19734]: cluster 2024-02-18T01:30:12.155558+0000 osd.1 (osd.1) 14 : cluster [DBG] 219.15 deep-scrub ok 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: cluster 2024-02-18T01:30:11.843165+0000 mgr.y (mgr.24475) 519 : cluster [DBG] pgmap v740: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: cluster 2024-02-18T01:30:12.018974+0000 mon.a (mon.0) 2373 : cluster [DBG] osdmap e421: 8 total, 8 up, 8 in 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: audit 2024-02-18T01:30:12.035426+0000 mgr.y (mgr.24475) 520 : audit [DBG] from='mon.? v1:172.21.15.155:0/619915283' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: audit 2024-02-18T01:30:12.035843+0000 mon.b (mon.1) 362 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "219.15"}]: dispatch 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: cluster 2024-02-18T01:30:12.150321+0000 osd.1 (osd.1) 13 : cluster [DBG] 219.15 deep-scrub starts 2024-02-18T01:30:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:13 smithi155 bash[20805]: cluster 2024-02-18T01:30:12.155558+0000 osd.1 (osd.1) 14 : cluster [DBG] 219.15 deep-scrub ok 2024-02-18T01:30:15.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:15 smithi105 bash[19734]: cluster 2024-02-18T01:30:13.845329+0000 mgr.y (mgr.24475) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:30:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:15 smithi105 bash[27669]: cluster 2024-02-18T01:30:13.845329+0000 mgr.y (mgr.24475) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:30:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:15 smithi155 bash[20805]: cluster 2024-02-18T01:30:13.845329+0000 mgr.y (mgr.24475) 521 : cluster [DBG] pgmap v742: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:30:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:16] "GET /metrics HTTP/1.1" 200 188923 "" "Prometheus/2.33.4" 2024-02-18T01:30:17.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:17 smithi105 bash[19734]: cluster 2024-02-18T01:30:15.846169+0000 mgr.y (mgr.24475) 522 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-18T01:30:17.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:17 smithi105 bash[27669]: cluster 2024-02-18T01:30:15.846169+0000 mgr.y (mgr.24475) 522 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-18T01:30:17.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:17 smithi155 bash[20805]: cluster 2024-02-18T01:30:15.846169+0000 mgr.y (mgr.24475) 522 : cluster [DBG] pgmap v743: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 852 B/s wr, 2 op/s 2024-02-18T01:30:18.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:17 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:19 smithi105 bash[19734]: cluster 2024-02-18T01:30:17.848323+0000 mgr.y (mgr.24475) 523 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 653 B/s wr, 2 op/s 2024-02-18T01:30:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:19 smithi105 bash[19734]: audit 2024-02-18T01:30:17.953475+0000 mgr.y (mgr.24475) 524 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:19 smithi105 bash[27669]: cluster 2024-02-18T01:30:17.848323+0000 mgr.y (mgr.24475) 523 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 653 B/s wr, 2 op/s 2024-02-18T01:30:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:19 smithi105 bash[27669]: audit 2024-02-18T01:30:17.953475+0000 mgr.y (mgr.24475) 524 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:19 smithi155 bash[20805]: cluster 2024-02-18T01:30:17.848323+0000 mgr.y (mgr.24475) 523 : cluster [DBG] pgmap v744: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 653 B/s wr, 2 op/s 2024-02-18T01:30:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:19 smithi155 bash[20805]: audit 2024-02-18T01:30:17.953475+0000 mgr.y (mgr.24475) 524 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:20 smithi105 bash[19734]: audit 2024-02-18T01:30:19.194919+0000 mon.a (mon.0) 2374 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:30:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:20 smithi105 bash[19734]: audit 2024-02-18T01:30:19.491130+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:20 smithi105 bash[27669]: audit 2024-02-18T01:30:19.194919+0000 mon.a (mon.0) 2374 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:30:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:20 smithi105 bash[27669]: audit 2024-02-18T01:30:19.491130+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:20 smithi155 bash[20805]: audit 2024-02-18T01:30:19.194919+0000 mon.a (mon.0) 2374 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:30:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:20 smithi155 bash[20805]: audit 2024-02-18T01:30:19.491130+0000 mon.a (mon.0) 2375 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:21 smithi105 bash[27669]: cluster 2024-02-18T01:30:19.849147+0000 mgr.y (mgr.24475) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 579 B/s wr, 2 op/s 2024-02-18T01:30:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:21 smithi105 bash[19734]: cluster 2024-02-18T01:30:19.849147+0000 mgr.y (mgr.24475) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 579 B/s wr, 2 op/s 2024-02-18T01:30:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:21 smithi155 bash[20805]: cluster 2024-02-18T01:30:19.849147+0000 mgr.y (mgr.24475) 525 : cluster [DBG] pgmap v745: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 579 B/s wr, 2 op/s 2024-02-18T01:30:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:23 smithi105 bash[19734]: cluster 2024-02-18T01:30:21.850072+0000 mgr.y (mgr.24475) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:30:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:23 smithi105 bash[27669]: cluster 2024-02-18T01:30:21.850072+0000 mgr.y (mgr.24475) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:30:23.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:23 smithi155 bash[20805]: cluster 2024-02-18T01:30:21.850072+0000 mgr.y (mgr.24475) 526 : cluster [DBG] pgmap v746: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:30:25.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:25 smithi105 bash[27669]: cluster 2024-02-18T01:30:23.852450+0000 mgr.y (mgr.24475) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 432 B/s wr, 2 op/s 2024-02-18T01:30:25.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:25 smithi105 bash[19734]: cluster 2024-02-18T01:30:23.852450+0000 mgr.y (mgr.24475) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 432 B/s wr, 2 op/s 2024-02-18T01:30:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:25 smithi155 bash[20805]: cluster 2024-02-18T01:30:23.852450+0000 mgr.y (mgr.24475) 527 : cluster [DBG] pgmap v747: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 432 B/s wr, 2 op/s 2024-02-18T01:30:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:26] "GET /metrics HTTP/1.1" 200 189057 "" "Prometheus/2.33.4" 2024-02-18T01:30:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[19734]: audit 2024-02-18T01:30:25.493289+0000 mon.a (mon.0) 2376 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[19734]: audit 2024-02-18T01:30:25.776074+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:30:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[19734]: audit 2024-02-18T01:30:25.777898+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:30:26.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[19734]: audit 2024-02-18T01:30:25.787639+0000 mon.a (mon.0) 2379 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[19734]: cluster 2024-02-18T01:30:25.853324+0000 mgr.y (mgr.24475) 528 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[27669]: audit 2024-02-18T01:30:25.493289+0000 mon.a (mon.0) 2376 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[27669]: audit 2024-02-18T01:30:25.776074+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[27669]: audit 2024-02-18T01:30:25.777898+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[27669]: audit 2024-02-18T01:30:25.787639+0000 mon.a (mon.0) 2379 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:26 smithi105 bash[27669]: cluster 2024-02-18T01:30:25.853324+0000 mgr.y (mgr.24475) 528 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:26 smithi155 bash[20805]: audit 2024-02-18T01:30:25.493289+0000 mon.a (mon.0) 2376 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:26 smithi155 bash[20805]: audit 2024-02-18T01:30:25.776074+0000 mon.a (mon.0) 2377 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:30:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:26 smithi155 bash[20805]: audit 2024-02-18T01:30:25.777898+0000 mon.a (mon.0) 2378 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:30:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:26 smithi155 bash[20805]: audit 2024-02-18T01:30:25.787639+0000 mon.a (mon.0) 2379 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:30:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:26 smithi155 bash[20805]: cluster 2024-02-18T01:30:25.853324+0000 mgr.y (mgr.24475) 528 : cluster [DBG] pgmap v748: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:28.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:27 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:28 smithi105 bash[19734]: cluster 2024-02-18T01:30:27.855473+0000 mgr.y (mgr.24475) 529 : cluster [DBG] pgmap v749: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:28 smithi105 bash[19734]: audit 2024-02-18T01:30:27.964734+0000 mgr.y (mgr.24475) 530 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:28 smithi105 bash[27669]: cluster 2024-02-18T01:30:27.855473+0000 mgr.y (mgr.24475) 529 : cluster [DBG] pgmap v749: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:28 smithi105 bash[27669]: audit 2024-02-18T01:30:27.964734+0000 mgr.y (mgr.24475) 530 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:28 smithi155 bash[20805]: cluster 2024-02-18T01:30:27.855473+0000 mgr.y (mgr.24475) 529 : cluster [DBG] pgmap v749: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:28 smithi155 bash[20805]: audit 2024-02-18T01:30:27.964734+0000 mgr.y (mgr.24475) 530 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:30 smithi105 bash[19734]: cluster 2024-02-18T01:30:29.856316+0000 mgr.y (mgr.24475) 531 : cluster [DBG] pgmap v750: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:30 smithi105 bash[27669]: cluster 2024-02-18T01:30:29.856316+0000 mgr.y (mgr.24475) 531 : cluster [DBG] pgmap v750: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:30 smithi155 bash[20805]: cluster 2024-02-18T01:30:29.856316+0000 mgr.y (mgr.24475) 531 : cluster [DBG] pgmap v750: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:32.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[24809]: debug 2024-02-18T01:30:32.063+0000 7fc8c465c700 -1 snap_mapper.add_oid found existing snaps mapped on 219:a8d5a5da:test-rados-api-smithi105-61833-76::foo:2, removing 2024-02-18T01:30:32.490 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[36150]: debug 2024-02-18T01:30:32.059+0000 7fcca5fa8700 -1 snap_mapper.add_oid found existing snaps mapped on 219:a8d5a5da:test-rados-api-smithi105-61833-76::foo:2, removing 2024-02-18T01:30:32.490 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[31001]: debug 2024-02-18T01:30:32.063+0000 7f4e8583f700 -1 snap_mapper.add_oid found existing snaps mapped on 219:a8d5a5da:test-rados-api-smithi105-61833-76::foo:2, removing 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[27669]: cluster 2024-02-18T01:30:31.857144+0000 mgr.y (mgr.24475) 532 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[27669]: audit 2024-02-18T01:30:32.068220+0000 mon.a (mon.0) 2380 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[27669]: audit 2024-02-18T01:30:32.068275+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[27669]: audit 2024-02-18T01:30:32.070051+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[27669]: audit 2024-02-18T01:30:32.070158+0000 mon.b (mon.1) 364 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[19734]: cluster 2024-02-18T01:30:31.857144+0000 mgr.y (mgr.24475) 532 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[19734]: audit 2024-02-18T01:30:32.068220+0000 mon.a (mon.0) 2380 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[19734]: audit 2024-02-18T01:30:32.068275+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[19734]: audit 2024-02-18T01:30:32.070051+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:33.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:32 smithi105 bash[19734]: audit 2024-02-18T01:30:32.070158+0000 mon.b (mon.1) 364 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[20805]: cluster 2024-02-18T01:30:31.857144+0000 mgr.y (mgr.24475) 532 : cluster [DBG] pgmap v751: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:30:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[20805]: audit 2024-02-18T01:30:32.068220+0000 mon.a (mon.0) 2380 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[20805]: audit 2024-02-18T01:30:32.068275+0000 mon.b (mon.1) 363 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[20805]: audit 2024-02-18T01:30:32.070051+0000 mon.a (mon.0) 2381 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:32 smithi155 bash[20805]: audit 2024-02-18T01:30:32.070158+0000 mon.b (mon.1) 364 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-75"}]: dispatch 2024-02-18T01:30:34.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:33 smithi105 bash[19734]: cluster 2024-02-18T01:30:32.938794+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-18T01:30:34.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:33 smithi105 bash[27669]: cluster 2024-02-18T01:30:32.938794+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-18T01:30:34.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:33 smithi155 bash[20805]: cluster 2024-02-18T01:30:32.938794+0000 mon.a (mon.0) 2382 : cluster [DBG] osdmap e422: 8 total, 8 up, 8 in 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: cluster 2024-02-18T01:30:33.858079+0000 mgr.y (mgr.24475) 533 : cluster [DBG] pgmap v753: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: cluster 2024-02-18T01:30:33.950608+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: audit 2024-02-18T01:30:33.954323+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: audit 2024-02-18T01:30:33.956827+0000 mon.a (mon.0) 2384 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: audit 2024-02-18T01:30:34.949709+0000 mon.a (mon.0) 2385 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: cluster 2024-02-18T01:30:34.950094+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-18T01:30:35.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[27669]: audit 2024-02-18T01:30:34.954711+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: cluster 2024-02-18T01:30:33.858079+0000 mgr.y (mgr.24475) 533 : cluster [DBG] pgmap v753: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: cluster 2024-02-18T01:30:33.950608+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: audit 2024-02-18T01:30:33.954323+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: audit 2024-02-18T01:30:33.956827+0000 mon.a (mon.0) 2384 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: audit 2024-02-18T01:30:34.949709+0000 mon.a (mon.0) 2385 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: cluster 2024-02-18T01:30:34.950094+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-18T01:30:35.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:34 smithi105 bash[19734]: audit 2024-02-18T01:30:34.954711+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: cluster 2024-02-18T01:30:33.858079+0000 mgr.y (mgr.24475) 533 : cluster [DBG] pgmap v753: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: cluster 2024-02-18T01:30:33.950608+0000 mon.a (mon.0) 2383 : cluster [DBG] osdmap e423: 8 total, 8 up, 8 in 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: audit 2024-02-18T01:30:33.954323+0000 mon.b (mon.1) 365 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: audit 2024-02-18T01:30:33.956827+0000 mon.a (mon.0) 2384 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: audit 2024-02-18T01:30:34.949709+0000 mon.a (mon.0) 2385 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:35.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: cluster 2024-02-18T01:30:34.950094+0000 mon.a (mon.0) 2386 : cluster [DBG] osdmap e424: 8 total, 8 up, 8 in 2024-02-18T01:30:35.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:34 smithi155 bash[20805]: audit 2024-02-18T01:30:34.954711+0000 mon.b (mon.1) 366 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: audit 2024-02-18T01:30:34.959496+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: audit 2024-02-18T01:30:34.960003+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: audit 2024-02-18T01:30:35.950685+0000 mon.a (mon.0) 2388 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: cluster 2024-02-18T01:30:35.950846+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: audit 2024-02-18T01:30:35.953686+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[19734]: audit 2024-02-18T01:30:35.954013+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: audit 2024-02-18T01:30:34.959496+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: audit 2024-02-18T01:30:34.960003+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: audit 2024-02-18T01:30:35.950685+0000 mon.a (mon.0) 2388 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: cluster 2024-02-18T01:30:35.950846+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-18T01:30:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: audit 2024-02-18T01:30:35.953686+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:35 smithi105 bash[27669]: audit 2024-02-18T01:30:35.954013+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: audit 2024-02-18T01:30:34.959496+0000 mon.b (mon.1) 367 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: audit 2024-02-18T01:30:34.960003+0000 mon.a (mon.0) 2387 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: audit 2024-02-18T01:30:35.950685+0000 mon.a (mon.0) 2388 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: cluster 2024-02-18T01:30:35.950846+0000 mon.a (mon.0) 2389 : cluster [DBG] osdmap e425: 8 total, 8 up, 8 in 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: audit 2024-02-18T01:30:35.953686+0000 mon.b (mon.1) 368 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:35 smithi155 bash[20805]: audit 2024-02-18T01:30:35.954013+0000 mon.a (mon.0) 2390 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:36.972 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:36] "GET /metrics HTTP/1.1" 200 189155 "" "Prometheus/2.33.4" 2024-02-18T01:30:36.973 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[19734]: cluster 2024-02-18T01:30:35.858948+0000 mgr.y (mgr.24475) 534 : cluster [DBG] pgmap v756: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:36.973 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[19734]: audit 2024-02-18T01:30:36.952390+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:36.973 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[19734]: cluster 2024-02-18T01:30:36.952539+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-18T01:30:36.973 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[19734]: audit 2024-02-18T01:30:36.954519+0000 mon.a (mon.0) 2393 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:36.973 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[19734]: audit 2024-02-18T01:30:36.954564+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:37.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[27669]: cluster 2024-02-18T01:30:35.858948+0000 mgr.y (mgr.24475) 534 : cluster [DBG] pgmap v756: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:37.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[27669]: audit 2024-02-18T01:30:36.952390+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:37.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[27669]: cluster 2024-02-18T01:30:36.952539+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-18T01:30:37.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[27669]: audit 2024-02-18T01:30:36.954519+0000 mon.a (mon.0) 2393 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:37.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:36 smithi105 bash[27669]: audit 2024-02-18T01:30:36.954564+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:36 smithi155 bash[20805]: cluster 2024-02-18T01:30:35.858948+0000 mgr.y (mgr.24475) 534 : cluster [DBG] pgmap v756: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:30:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:36 smithi155 bash[20805]: audit 2024-02-18T01:30:36.952390+0000 mon.a (mon.0) 2391 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:36 smithi155 bash[20805]: cluster 2024-02-18T01:30:36.952539+0000 mon.a (mon.0) 2392 : cluster [DBG] osdmap e426: 8 total, 8 up, 8 in 2024-02-18T01:30:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:36 smithi155 bash[20805]: audit 2024-02-18T01:30:36.954519+0000 mon.a (mon.0) 2393 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:37.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:36 smithi155 bash[20805]: audit 2024-02-18T01:30:36.954564+0000 mon.b (mon.1) 369 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:38.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:37 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: cluster 2024-02-18T01:30:37.861012+0000 mgr.y (mgr.24475) 535 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: audit 2024-02-18T01:30:37.957143+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: cluster 2024-02-18T01:30:37.957364+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: audit 2024-02-18T01:30:37.959011+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: audit 2024-02-18T01:30:37.960018+0000 mon.a (mon.0) 2396 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[19734]: audit 2024-02-18T01:30:37.969851+0000 mgr.y (mgr.24475) 536 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:39.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: cluster 2024-02-18T01:30:37.861012+0000 mgr.y (mgr.24475) 535 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: audit 2024-02-18T01:30:37.957143+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: cluster 2024-02-18T01:30:37.957364+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-18T01:30:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: audit 2024-02-18T01:30:37.959011+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: audit 2024-02-18T01:30:37.960018+0000 mon.a (mon.0) 2396 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:38 smithi105 bash[27669]: audit 2024-02-18T01:30:37.969851+0000 mgr.y (mgr.24475) 536 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: cluster 2024-02-18T01:30:37.861012+0000 mgr.y (mgr.24475) 535 : cluster [DBG] pgmap v759: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: audit 2024-02-18T01:30:37.957143+0000 mon.a (mon.0) 2394 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: cluster 2024-02-18T01:30:37.957364+0000 mon.a (mon.0) 2395 : cluster [DBG] osdmap e427: 8 total, 8 up, 8 in 2024-02-18T01:30:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: audit 2024-02-18T01:30:37.959011+0000 mon.b (mon.1) 370 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: audit 2024-02-18T01:30:37.960018+0000 mon.a (mon.0) 2396 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:39.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:38 smithi155 bash[20805]: audit 2024-02-18T01:30:37.969851+0000 mgr.y (mgr.24475) 536 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:39 smithi105 bash[19734]: audit 2024-02-18T01:30:38.958049+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:39 smithi105 bash[19734]: cluster 2024-02-18T01:30:38.958139+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:39 smithi105 bash[19734]: audit 2024-02-18T01:30:38.986900+0000 mon.a (mon.0) 2399 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: audit 2024-02-18T01:30:38.987156+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:38.958049+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: cluster 2024-02-18T01:30:38.958139+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-18T01:30:40.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:38.986900+0000 mon.a (mon.0) 2399 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:40.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:38.987156+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:40.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:39 smithi155 bash[20805]: audit 2024-02-18T01:30:38.958049+0000 mon.a (mon.0) 2397 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:40.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:39 smithi155 bash[20805]: cluster 2024-02-18T01:30:38.958139+0000 mon.a (mon.0) 2398 : cluster [DBG] osdmap e428: 8 total, 8 up, 8 in 2024-02-18T01:30:40.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:39 smithi155 bash[20805]: audit 2024-02-18T01:30:38.986900+0000 mon.a (mon.0) 2399 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:40.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:39 smithi155 bash[20805]: audit 2024-02-18T01:30:38.987156+0000 mon.b (mon.1) 371 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-02-18T01:30:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: cluster 2024-02-18T01:30:39.861854+0000 mgr.y (mgr.24475) 537 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: audit 2024-02-18T01:30:39.981041+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T01:30:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: cluster 2024-02-18T01:30:39.981178+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-18T01:30:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: audit 2024-02-18T01:30:40.001979+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:41.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[19734]: audit 2024-02-18T01:30:40.004767+0000 mon.a (mon.0) 2402 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: cluster 2024-02-18T01:30:39.861854+0000 mgr.y (mgr.24475) 537 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:39.981041+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: cluster 2024-02-18T01:30:39.981178+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-18T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:40.001979+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:41.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:40 smithi105 bash[27669]: audit 2024-02-18T01:30:40.004767+0000 mon.a (mon.0) 2402 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:40 smithi155 bash[20805]: cluster 2024-02-18T01:30:39.861854+0000 mgr.y (mgr.24475) 537 : cluster [DBG] pgmap v762: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:40 smithi155 bash[20805]: audit 2024-02-18T01:30:39.981041+0000 mon.a (mon.0) 2400 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-02-18T01:30:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:40 smithi155 bash[20805]: cluster 2024-02-18T01:30:39.981178+0000 mon.a (mon.0) 2401 : cluster [DBG] osdmap e429: 8 total, 8 up, 8 in 2024-02-18T01:30:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:40 smithi155 bash[20805]: audit 2024-02-18T01:30:40.001979+0000 mon.b (mon.1) 372 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:41.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:40 smithi155 bash[20805]: audit 2024-02-18T01:30:40.004767+0000 mon.a (mon.0) 2402 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-02-18T01:30:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[19734]: audit 2024-02-18T01:30:40.991981+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T01:30:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[19734]: cluster 2024-02-18T01:30:40.992123+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-18T01:30:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[19734]: audit 2024-02-18T01:30:41.011558+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:42.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[19734]: audit 2024-02-18T01:30:41.011818+0000 mon.a (mon.0) 2405 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:42.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[27669]: audit 2024-02-18T01:30:40.991981+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T01:30:42.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[27669]: cluster 2024-02-18T01:30:40.992123+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-18T01:30:42.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[27669]: audit 2024-02-18T01:30:41.011558+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:42.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:41 smithi105 bash[27669]: audit 2024-02-18T01:30:41.011818+0000 mon.a (mon.0) 2405 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:42 smithi155 bash[20805]: audit 2024-02-18T01:30:40.991981+0000 mon.a (mon.0) 2403 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-02-18T01:30:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:42 smithi155 bash[20805]: cluster 2024-02-18T01:30:40.992123+0000 mon.a (mon.0) 2404 : cluster [DBG] osdmap e430: 8 total, 8 up, 8 in 2024-02-18T01:30:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:42 smithi155 bash[20805]: audit 2024-02-18T01:30:41.011558+0000 mon.b (mon.1) 373 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:42 smithi155 bash[20805]: audit 2024-02-18T01:30:41.011818+0000 mon.a (mon.0) 2405 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: cluster 2024-02-18T01:30:41.862690+0000 mgr.y (mgr.24475) 538 : cluster [DBG] pgmap v765: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: audit 2024-02-18T01:30:42.003684+0000 mon.a (mon.0) 2406 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: cluster 2024-02-18T01:30:42.003830+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: audit 2024-02-18T01:30:42.046765+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: audit 2024-02-18T01:30:42.046961+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: audit 2024-02-18T01:30:42.047802+0000 mon.a (mon.0) 2409 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[19734]: audit 2024-02-18T01:30:42.048177+0000 mon.b (mon.1) 375 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: cluster 2024-02-18T01:30:41.862690+0000 mgr.y (mgr.24475) 538 : cluster [DBG] pgmap v765: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: audit 2024-02-18T01:30:42.003684+0000 mon.a (mon.0) 2406 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: cluster 2024-02-18T01:30:42.003830+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: audit 2024-02-18T01:30:42.046765+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: audit 2024-02-18T01:30:42.046961+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: audit 2024-02-18T01:30:42.047802+0000 mon.a (mon.0) 2409 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:43 smithi105 bash[27669]: audit 2024-02-18T01:30:42.048177+0000 mon.b (mon.1) 375 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: cluster 2024-02-18T01:30:41.862690+0000 mgr.y (mgr.24475) 538 : cluster [DBG] pgmap v765: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: audit 2024-02-18T01:30:42.003684+0000 mon.a (mon.0) 2406 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-77","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: cluster 2024-02-18T01:30:42.003830+0000 mon.a (mon.0) 2407 : cluster [DBG] osdmap e431: 8 total, 8 up, 8 in 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: audit 2024-02-18T01:30:42.046765+0000 mon.a (mon.0) 2408 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: audit 2024-02-18T01:30:42.046961+0000 mon.b (mon.1) 374 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: audit 2024-02-18T01:30:42.047802+0000 mon.a (mon.0) 2409 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:43.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:43 smithi155 bash[20805]: audit 2024-02-18T01:30:42.048177+0000 mon.b (mon.1) 375 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-77"}]: dispatch 2024-02-18T01:30:44.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:44 smithi105 bash[27669]: cluster 2024-02-18T01:30:43.006878+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-18T01:30:44.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:44 smithi105 bash[19734]: cluster 2024-02-18T01:30:43.006878+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-18T01:30:44.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:44 smithi155 bash[20805]: cluster 2024-02-18T01:30:43.006878+0000 mon.a (mon.0) 2410 : cluster [DBG] osdmap e432: 8 total, 8 up, 8 in 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[19734]: cluster 2024-02-18T01:30:43.863271+0000 mgr.y (mgr.24475) 539 : cluster [DBG] pgmap v768: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[19734]: cluster 2024-02-18T01:30:44.030190+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[19734]: audit 2024-02-18T01:30:44.039569+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[19734]: audit 2024-02-18T01:30:44.039736+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[27669]: cluster 2024-02-18T01:30:43.863271+0000 mgr.y (mgr.24475) 539 : cluster [DBG] pgmap v768: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[27669]: cluster 2024-02-18T01:30:44.030190+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-18T01:30:45.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[27669]: audit 2024-02-18T01:30:44.039569+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:45.493 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:45 smithi105 bash[27669]: audit 2024-02-18T01:30:44.039736+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:45 smithi155 bash[20805]: cluster 2024-02-18T01:30:43.863271+0000 mgr.y (mgr.24475) 539 : cluster [DBG] pgmap v768: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:45 smithi155 bash[20805]: cluster 2024-02-18T01:30:44.030190+0000 mon.a (mon.0) 2411 : cluster [DBG] osdmap e433: 8 total, 8 up, 8 in 2024-02-18T01:30:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:45 smithi155 bash[20805]: audit 2024-02-18T01:30:44.039569+0000 mon.b (mon.1) 376 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:45 smithi155 bash[20805]: audit 2024-02-18T01:30:44.039736+0000 mon.a (mon.0) 2412 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[27669]: audit 2024-02-18T01:30:45.034211+0000 mon.a (mon.0) 2413 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[27669]: cluster 2024-02-18T01:30:45.034491+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[27669]: audit 2024-02-18T01:30:45.050060+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[27669]: audit 2024-02-18T01:30:45.053737+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[27669]: audit 2024-02-18T01:30:45.054186+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[19734]: audit 2024-02-18T01:30:45.034211+0000 mon.a (mon.0) 2413 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[19734]: cluster 2024-02-18T01:30:45.034491+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-18T01:30:46.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[19734]: audit 2024-02-18T01:30:45.050060+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:46.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[19734]: audit 2024-02-18T01:30:45.053737+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[19734]: audit 2024-02-18T01:30:45.054186+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:46 smithi155 bash[20805]: audit 2024-02-18T01:30:45.034211+0000 mon.a (mon.0) 2413 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:46 smithi155 bash[20805]: cluster 2024-02-18T01:30:45.034491+0000 mon.a (mon.0) 2414 : cluster [DBG] osdmap e434: 8 total, 8 up, 8 in 2024-02-18T01:30:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:46 smithi155 bash[20805]: audit 2024-02-18T01:30:45.050060+0000 mon.b (mon.1) 377 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:46 smithi155 bash[20805]: audit 2024-02-18T01:30:45.053737+0000 mon.b (mon.1) 378 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:46 smithi155 bash[20805]: audit 2024-02-18T01:30:45.054186+0000 mon.a (mon.0) 2415 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:46] "GET /metrics HTTP/1.1" 200 189155 "" "Prometheus/2.33.4" 2024-02-18T01:30:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[27669]: cluster 2024-02-18T01:30:45.864013+0000 mgr.y (mgr.24475) 540 : cluster [DBG] pgmap v771: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[27669]: audit 2024-02-18T01:30:46.048996+0000 mon.a (mon.0) 2416 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[27669]: cluster 2024-02-18T01:30:46.049101+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-18T01:30:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[27669]: audit 2024-02-18T01:30:46.052940+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[27669]: audit 2024-02-18T01:30:46.057336+0000 mon.a (mon.0) 2418 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[19734]: cluster 2024-02-18T01:30:45.864013+0000 mgr.y (mgr.24475) 540 : cluster [DBG] pgmap v771: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[19734]: audit 2024-02-18T01:30:46.048996+0000 mon.a (mon.0) 2416 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[19734]: cluster 2024-02-18T01:30:46.049101+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-18T01:30:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[19734]: audit 2024-02-18T01:30:46.052940+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:47.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:47 smithi105 bash[19734]: audit 2024-02-18T01:30:46.057336+0000 mon.a (mon.0) 2418 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[20805]: cluster 2024-02-18T01:30:45.864013+0000 mgr.y (mgr.24475) 540 : cluster [DBG] pgmap v771: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-02-18T01:30:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[20805]: audit 2024-02-18T01:30:46.048996+0000 mon.a (mon.0) 2416 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[20805]: cluster 2024-02-18T01:30:46.049101+0000 mon.a (mon.0) 2417 : cluster [DBG] osdmap e435: 8 total, 8 up, 8 in 2024-02-18T01:30:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[20805]: audit 2024-02-18T01:30:46.052940+0000 mon.b (mon.1) 379 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:47.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[20805]: audit 2024-02-18T01:30:46.057336+0000 mon.a (mon.0) 2418 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:48.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:47 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:48.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:48 smithi155 bash[20805]: audit 2024-02-18T01:30:47.067833+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:48.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:48 smithi155 bash[20805]: cluster 2024-02-18T01:30:47.068055+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-18T01:30:48.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:48 smithi155 bash[20805]: audit 2024-02-18T01:30:47.071949+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:48.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:48 smithi155 bash[20805]: audit 2024-02-18T01:30:47.072010+0000 mon.a (mon.0) 2421 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[27669]: audit 2024-02-18T01:30:47.067833+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[27669]: cluster 2024-02-18T01:30:47.068055+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[27669]: audit 2024-02-18T01:30:47.071949+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[27669]: audit 2024-02-18T01:30:47.072010+0000 mon.a (mon.0) 2421 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[19734]: audit 2024-02-18T01:30:47.067833+0000 mon.a (mon.0) 2419 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[19734]: cluster 2024-02-18T01:30:47.068055+0000 mon.a (mon.0) 2420 : cluster [DBG] osdmap e436: 8 total, 8 up, 8 in 2024-02-18T01:30:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[19734]: audit 2024-02-18T01:30:47.071949+0000 mon.b (mon.1) 380 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:48.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:48 smithi105 bash[19734]: audit 2024-02-18T01:30:47.072010+0000 mon.a (mon.0) 2421 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: cluster 2024-02-18T01:30:47.866196+0000 mgr.y (mgr.24475) 541 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: audit 2024-02-18T01:30:47.980861+0000 mgr.y (mgr.24475) 542 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: audit 2024-02-18T01:30:48.074470+0000 mon.a (mon.0) 2422 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: cluster 2024-02-18T01:30:48.074692+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: audit 2024-02-18T01:30:48.077135+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: audit 2024-02-18T01:30:48.077276+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: audit 2024-02-18T01:30:49.076734+0000 mon.a (mon.0) 2425 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[27669]: cluster 2024-02-18T01:30:49.076941+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: cluster 2024-02-18T01:30:47.866196+0000 mgr.y (mgr.24475) 541 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: audit 2024-02-18T01:30:47.980861+0000 mgr.y (mgr.24475) 542 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: audit 2024-02-18T01:30:48.074470+0000 mon.a (mon.0) 2422 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:49.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: cluster 2024-02-18T01:30:48.074692+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-18T01:30:49.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: audit 2024-02-18T01:30:48.077135+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: audit 2024-02-18T01:30:48.077276+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: audit 2024-02-18T01:30:49.076734+0000 mon.a (mon.0) 2425 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:49.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:49 smithi105 bash[19734]: cluster 2024-02-18T01:30:49.076941+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-18T01:30:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: cluster 2024-02-18T01:30:47.866196+0000 mgr.y (mgr.24475) 541 : cluster [DBG] pgmap v774: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: audit 2024-02-18T01:30:47.980861+0000 mgr.y (mgr.24475) 542 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: audit 2024-02-18T01:30:48.074470+0000 mon.a (mon.0) 2422 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: cluster 2024-02-18T01:30:48.074692+0000 mon.a (mon.0) 2423 : cluster [DBG] osdmap e437: 8 total, 8 up, 8 in 2024-02-18T01:30:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: audit 2024-02-18T01:30:48.077135+0000 mon.a (mon.0) 2424 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: audit 2024-02-18T01:30:48.077276+0000 mon.b (mon.1) 381 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: audit 2024-02-18T01:30:49.076734+0000 mon.a (mon.0) 2425 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-79","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:49.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:49 smithi155 bash[20805]: cluster 2024-02-18T01:30:49.076941+0000 mon.a (mon.0) 2426 : cluster [DBG] osdmap e438: 8 total, 8 up, 8 in 2024-02-18T01:30:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:50 smithi105 bash[19734]: cluster 2024-02-18T01:30:49.867040+0000 mgr.y (mgr.24475) 543 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:50 smithi105 bash[27669]: cluster 2024-02-18T01:30:49.867040+0000 mgr.y (mgr.24475) 543 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:50 smithi155 bash[20805]: cluster 2024-02-18T01:30:49.867040+0000 mgr.y (mgr.24475) 543 : cluster [DBG] pgmap v777: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:51 smithi105 bash[27669]: cluster 2024-02-18T01:30:50.093138+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-18T01:30:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:51 smithi105 bash[19734]: cluster 2024-02-18T01:30:50.093138+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-18T01:30:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:51 smithi155 bash[20805]: cluster 2024-02-18T01:30:50.093138+0000 mon.a (mon.0) 2427 : cluster [DBG] osdmap e439: 8 total, 8 up, 8 in 2024-02-18T01:30:52.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:30:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: cluster 2024-02-18T01:30:51.109803+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: audit 2024-02-18T01:30:51.152888+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: audit 2024-02-18T01:30:51.153046+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: audit 2024-02-18T01:30:51.154037+0000 mon.a (mon.0) 2430 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: audit 2024-02-18T01:30:51.154335+0000 mon.b (mon.1) 383 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:52 smithi155 bash[20805]: cluster 2024-02-18T01:30:51.867791+0000 mgr.y (mgr.24475) 544 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: cluster 2024-02-18T01:30:51.109803+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: audit 2024-02-18T01:30:51.152888+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: audit 2024-02-18T01:30:51.153046+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: audit 2024-02-18T01:30:51.154037+0000 mon.a (mon.0) 2430 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: audit 2024-02-18T01:30:51.154335+0000 mon.b (mon.1) 383 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[19734]: cluster 2024-02-18T01:30:51.867791+0000 mgr.y (mgr.24475) 544 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: cluster 2024-02-18T01:30:51.109803+0000 mon.a (mon.0) 2428 : cluster [DBG] osdmap e440: 8 total, 8 up, 8 in 2024-02-18T01:30:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: audit 2024-02-18T01:30:51.152888+0000 mon.a (mon.0) 2429 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: audit 2024-02-18T01:30:51.153046+0000 mon.b (mon.1) 382 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: audit 2024-02-18T01:30:51.154037+0000 mon.a (mon.0) 2430 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: audit 2024-02-18T01:30:51.154335+0000 mon.b (mon.1) 383 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-79"}]: dispatch 2024-02-18T01:30:52.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:52 smithi105 bash[27669]: cluster 2024-02-18T01:30:51.867791+0000 mgr.y (mgr.24475) 544 : cluster [DBG] pgmap v780: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:30:53.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:53 smithi105 bash[19734]: cluster 2024-02-18T01:30:52.117660+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-18T01:30:53.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:53 smithi105 bash[27669]: cluster 2024-02-18T01:30:52.117660+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-18T01:30:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:53 smithi155 bash[20805]: cluster 2024-02-18T01:30:52.117660+0000 mon.a (mon.0) 2431 : cluster [DBG] osdmap e441: 8 total, 8 up, 8 in 2024-02-18T01:30:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[27669]: cluster 2024-02-18T01:30:53.132007+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-18T01:30:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[27669]: audit 2024-02-18T01:30:53.134595+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[27669]: audit 2024-02-18T01:30:53.135614+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[27669]: cluster 2024-02-18T01:30:53.868664+0000 mgr.y (mgr.24475) 545 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:54.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[19734]: cluster 2024-02-18T01:30:53.132007+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-18T01:30:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[19734]: audit 2024-02-18T01:30:53.134595+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[19734]: audit 2024-02-18T01:30:53.135614+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:54 smithi105 bash[19734]: cluster 2024-02-18T01:30:53.868664+0000 mgr.y (mgr.24475) 545 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:54 smithi155 bash[20805]: cluster 2024-02-18T01:30:53.132007+0000 mon.a (mon.0) 2432 : cluster [DBG] osdmap e442: 8 total, 8 up, 8 in 2024-02-18T01:30:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:54 smithi155 bash[20805]: audit 2024-02-18T01:30:53.134595+0000 mon.b (mon.1) 384 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:54 smithi155 bash[20805]: audit 2024-02-18T01:30:53.135614+0000 mon.a (mon.0) 2433 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:30:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:54 smithi155 bash[20805]: cluster 2024-02-18T01:30:53.868664+0000 mgr.y (mgr.24475) 545 : cluster [DBG] pgmap v783: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:54.137634+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: cluster 2024-02-18T01:30:54.137994+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:54.140042+0000 mon.b (mon.1) 385 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:54.143604+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:54.148144+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:55.141881+0000 mon.a (mon.0) 2437 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: cluster 2024-02-18T01:30:55.142118+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:55.143811+0000 mon.a (mon.0) 2439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:55.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[27669]: audit 2024-02-18T01:30:55.143969+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:54.137634+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: cluster 2024-02-18T01:30:54.137994+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:54.140042+0000 mon.b (mon.1) 385 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:54.143604+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:54.148144+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:55.141881+0000 mon.a (mon.0) 2437 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: cluster 2024-02-18T01:30:55.142118+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:55.143811+0000 mon.a (mon.0) 2439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:55.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:55 smithi105 bash[19734]: audit 2024-02-18T01:30:55.143969+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:54.137634+0000 mon.a (mon.0) 2434 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: cluster 2024-02-18T01:30:54.137994+0000 mon.a (mon.0) 2435 : cluster [DBG] osdmap e443: 8 total, 8 up, 8 in 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:54.140042+0000 mon.b (mon.1) 385 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:54.143604+0000 mon.b (mon.1) 386 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:54.148144+0000 mon.a (mon.0) 2436 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:55.141881+0000 mon.a (mon.0) 2437 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:30:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: cluster 2024-02-18T01:30:55.142118+0000 mon.a (mon.0) 2438 : cluster [DBG] osdmap e444: 8 total, 8 up, 8 in 2024-02-18T01:30:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:55.143811+0000 mon.a (mon.0) 2439 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:55.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:55 smithi155 bash[20805]: audit 2024-02-18T01:30:55.143969+0000 mon.b (mon.1) 387 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[19734]: cluster 2024-02-18T01:30:55.869305+0000 mgr.y (mgr.24475) 546 : cluster [DBG] pgmap v786: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[19734]: audit 2024-02-18T01:30:56.144331+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[19734]: cluster 2024-02-18T01:30:56.144558+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[19734]: audit 2024-02-18T01:30:56.146216+0000 mon.a (mon.0) 2442 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[19734]: audit 2024-02-18T01:30:56.146256+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[27669]: cluster 2024-02-18T01:30:55.869305+0000 mgr.y (mgr.24475) 546 : cluster [DBG] pgmap v786: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:56.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[27669]: audit 2024-02-18T01:30:56.144331+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:56.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[27669]: cluster 2024-02-18T01:30:56.144558+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-18T01:30:56.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[27669]: audit 2024-02-18T01:30:56.146216+0000 mon.a (mon.0) 2442 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[27669]: audit 2024-02-18T01:30:56.146256+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:56 smithi155 bash[20805]: cluster 2024-02-18T01:30:55.869305+0000 mgr.y (mgr.24475) 546 : cluster [DBG] pgmap v786: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s 2024-02-18T01:30:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:56 smithi155 bash[20805]: audit 2024-02-18T01:30:56.144331+0000 mon.a (mon.0) 2440 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_tier","val": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:30:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:56 smithi155 bash[20805]: cluster 2024-02-18T01:30:56.144558+0000 mon.a (mon.0) 2441 : cluster [DBG] osdmap e445: 8 total, 8 up, 8 in 2024-02-18T01:30:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:56 smithi155 bash[20805]: audit 2024-02-18T01:30:56.146216+0000 mon.a (mon.0) 2442 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:56 smithi155 bash[20805]: audit 2024-02-18T01:30:56.146256+0000 mon.b (mon.1) 388 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:30:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:30:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:30:56] "GET /metrics HTTP/1.1" 200 189297 "" "Prometheus/2.33.4" 2024-02-18T01:30:58.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:30:57 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: audit 2024-02-18T01:30:57.147603+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: cluster 2024-02-18T01:30:57.147809+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: audit 2024-02-18T01:30:57.149470+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: audit 2024-02-18T01:30:57.149674+0000 mon.a (mon.0) 2445 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: cluster 2024-02-18T01:30:57.871287+0000 mgr.y (mgr.24475) 547 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:58.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:58 smithi155 bash[20805]: audit 2024-02-18T01:30:57.988487+0000 mgr.y (mgr.24475) 548 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: audit 2024-02-18T01:30:57.147603+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: cluster 2024-02-18T01:30:57.147809+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: audit 2024-02-18T01:30:57.149470+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: audit 2024-02-18T01:30:57.149674+0000 mon.a (mon.0) 2445 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: cluster 2024-02-18T01:30:57.871287+0000 mgr.y (mgr.24475) 547 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[19734]: audit 2024-02-18T01:30:57.988487+0000 mgr.y (mgr.24475) 548 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:58.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: audit 2024-02-18T01:30:57.147603+0000 mon.a (mon.0) 2443 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:30:58.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: cluster 2024-02-18T01:30:57.147809+0000 mon.a (mon.0) 2444 : cluster [DBG] osdmap e446: 8 total, 8 up, 8 in 2024-02-18T01:30:58.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: audit 2024-02-18T01:30:57.149470+0000 mon.b (mon.1) 389 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: audit 2024-02-18T01:30:57.149674+0000 mon.a (mon.0) 2445 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:30:58.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: cluster 2024-02-18T01:30:57.871287+0000 mgr.y (mgr.24475) 547 : cluster [DBG] pgmap v789: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:30:58.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:58 smithi105 bash[27669]: audit 2024-02-18T01:30:57.988487+0000 mgr.y (mgr.24475) 548 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:30:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:59 smithi105 bash[27669]: audit 2024-02-18T01:30:58.157359+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:30:59 smithi105 bash[27669]: cluster 2024-02-18T01:30:58.157469+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-18T01:30:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:59 smithi105 bash[19734]: audit 2024-02-18T01:30:58.157359+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:59.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:30:59 smithi105 bash[19734]: cluster 2024-02-18T01:30:58.157469+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-18T01:30:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:59 smithi155 bash[20805]: audit 2024-02-18T01:30:58.157359+0000 mon.a (mon.0) 2446 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:30:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:30:59 smithi155 bash[20805]: cluster 2024-02-18T01:30:58.157469+0000 mon.a (mon.0) 2447 : cluster [DBG] osdmap e447: 8 total, 8 up, 8 in 2024-02-18T01:31:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:59.159828+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-18T01:31:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:00 smithi105 bash[27669]: cluster 2024-02-18T01:30:59.872124+0000 mgr.y (mgr.24475) 549 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:59.159828+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-18T01:31:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:00 smithi105 bash[19734]: cluster 2024-02-18T01:30:59.872124+0000 mgr.y (mgr.24475) 549 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:59.159828+0000 mon.a (mon.0) 2448 : cluster [DBG] osdmap e448: 8 total, 8 up, 8 in 2024-02-18T01:31:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:00 smithi155 bash[20805]: cluster 2024-02-18T01:30:59.872124+0000 mgr.y (mgr.24475) 549 : cluster [DBG] pgmap v792: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[19734]: cluster 2024-02-18T01:31:00.170974+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[19734]: audit 2024-02-18T01:31:00.210093+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[19734]: audit 2024-02-18T01:31:00.210276+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[19734]: audit 2024-02-18T01:31:00.211177+0000 mon.a (mon.0) 2451 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[19734]: audit 2024-02-18T01:31:00.211504+0000 mon.b (mon.1) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[27669]: cluster 2024-02-18T01:31:00.170974+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-18T01:31:01.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[27669]: audit 2024-02-18T01:31:00.210093+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[27669]: audit 2024-02-18T01:31:00.210276+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[27669]: audit 2024-02-18T01:31:00.211177+0000 mon.a (mon.0) 2451 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:01.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:01 smithi105 bash[27669]: audit 2024-02-18T01:31:00.211504+0000 mon.b (mon.1) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[20805]: cluster 2024-02-18T01:31:00.170974+0000 mon.a (mon.0) 2449 : cluster [DBG] osdmap e449: 8 total, 8 up, 8 in 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[20805]: audit 2024-02-18T01:31:00.210093+0000 mon.a (mon.0) 2450 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[20805]: audit 2024-02-18T01:31:00.210276+0000 mon.b (mon.1) 390 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[20805]: audit 2024-02-18T01:31:00.211177+0000 mon.a (mon.0) 2451 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:01 smithi155 bash[20805]: audit 2024-02-18T01:31:00.211504+0000 mon.b (mon.1) 391 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-81"}]: dispatch 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[19734]: cluster 2024-02-18T01:31:01.184134+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[19734]: cluster 2024-02-18T01:31:01.872919+0000 mgr.y (mgr.24475) 550 : cluster [DBG] pgmap v795: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[19734]: cluster 2024-02-18T01:31:02.186364+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[19734]: audit 2024-02-18T01:31:02.190156+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[19734]: audit 2024-02-18T01:31:02.196507+0000 mon.a (mon.0) 2454 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[27669]: cluster 2024-02-18T01:31:01.184134+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-18T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[27669]: cluster 2024-02-18T01:31:01.872919+0000 mgr.y (mgr.24475) 550 : cluster [DBG] pgmap v795: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[27669]: cluster 2024-02-18T01:31:02.186364+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-18T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[27669]: audit 2024-02-18T01:31:02.190156+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:02.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:02 smithi105 bash[27669]: audit 2024-02-18T01:31:02.196507+0000 mon.a (mon.0) 2454 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:02 smithi155 bash[20805]: cluster 2024-02-18T01:31:01.184134+0000 mon.a (mon.0) 2452 : cluster [DBG] osdmap e450: 8 total, 8 up, 8 in 2024-02-18T01:31:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:02 smithi155 bash[20805]: cluster 2024-02-18T01:31:01.872919+0000 mgr.y (mgr.24475) 550 : cluster [DBG] pgmap v795: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:02 smithi155 bash[20805]: cluster 2024-02-18T01:31:02.186364+0000 mon.a (mon.0) 2453 : cluster [DBG] osdmap e451: 8 total, 8 up, 8 in 2024-02-18T01:31:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:02 smithi155 bash[20805]: audit 2024-02-18T01:31:02.190156+0000 mon.b (mon.1) 392 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:02 smithi155 bash[20805]: audit 2024-02-18T01:31:02.196507+0000 mon.a (mon.0) 2454 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[27669]: audit 2024-02-18T01:31:03.192752+0000 mon.a (mon.0) 2455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[27669]: cluster 2024-02-18T01:31:03.192981+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[27669]: audit 2024-02-18T01:31:03.196494+0000 mon.b (mon.1) 393 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[27669]: cluster 2024-02-18T01:31:03.873778+0000 mgr.y (mgr.24475) 551 : cluster [DBG] pgmap v798: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[19734]: audit 2024-02-18T01:31:03.192752+0000 mon.a (mon.0) 2455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[19734]: cluster 2024-02-18T01:31:03.192981+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-18T01:31:04.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[19734]: audit 2024-02-18T01:31:03.196494+0000 mon.b (mon.1) 393 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:04.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:04 smithi105 bash[19734]: cluster 2024-02-18T01:31:03.873778+0000 mgr.y (mgr.24475) 551 : cluster [DBG] pgmap v798: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:04 smithi155 bash[20805]: audit 2024-02-18T01:31:03.192752+0000 mon.a (mon.0) 2455 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:04 smithi155 bash[20805]: cluster 2024-02-18T01:31:03.192981+0000 mon.a (mon.0) 2456 : cluster [DBG] osdmap e452: 8 total, 8 up, 8 in 2024-02-18T01:31:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:04 smithi155 bash[20805]: audit 2024-02-18T01:31:03.196494+0000 mon.b (mon.1) 393 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:04 smithi155 bash[20805]: cluster 2024-02-18T01:31:03.873778+0000 mgr.y (mgr.24475) 551 : cluster [DBG] pgmap v798: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[27669]: cluster 2024-02-18T01:31:04.196239+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[27669]: audit 2024-02-18T01:31:04.200012+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[27669]: audit 2024-02-18T01:31:04.201618+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[27669]: audit 2024-02-18T01:31:05.196638+0000 mon.a (mon.0) 2459 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[27669]: cluster 2024-02-18T01:31:05.196813+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[19734]: cluster 2024-02-18T01:31:04.196239+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-18T01:31:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[19734]: audit 2024-02-18T01:31:04.200012+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[19734]: audit 2024-02-18T01:31:04.201618+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[19734]: audit 2024-02-18T01:31:05.196638+0000 mon.a (mon.0) 2459 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:05.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:05 smithi105 bash[19734]: cluster 2024-02-18T01:31:05.196813+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-18T01:31:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:05 smithi155 bash[20805]: cluster 2024-02-18T01:31:04.196239+0000 mon.a (mon.0) 2457 : cluster [DBG] osdmap e453: 8 total, 8 up, 8 in 2024-02-18T01:31:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:05 smithi155 bash[20805]: audit 2024-02-18T01:31:04.200012+0000 mon.b (mon.1) 394 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:05 smithi155 bash[20805]: audit 2024-02-18T01:31:04.201618+0000 mon.a (mon.0) 2458 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:05 smithi155 bash[20805]: audit 2024-02-18T01:31:05.196638+0000 mon.a (mon.0) 2459 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:05 smithi155 bash[20805]: cluster 2024-02-18T01:31:05.196813+0000 mon.a (mon.0) 2460 : cluster [DBG] osdmap e454: 8 total, 8 up, 8 in 2024-02-18T01:31:06.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: audit 2024-02-18T01:31:05.203287+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: audit 2024-02-18T01:31:05.203716+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: cluster 2024-02-18T01:31:05.874638+0000 mgr.y (mgr.24475) 552 : cluster [DBG] pgmap v801: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: audit 2024-02-18T01:31:06.198831+0000 mon.a (mon.0) 2462 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]': finished 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: cluster 2024-02-18T01:31:06.199086+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: audit 2024-02-18T01:31:06.200573+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[27669]: audit 2024-02-18T01:31:06.200649+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: audit 2024-02-18T01:31:05.203287+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: audit 2024-02-18T01:31:05.203716+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: cluster 2024-02-18T01:31:05.874638+0000 mgr.y (mgr.24475) 552 : cluster [DBG] pgmap v801: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: audit 2024-02-18T01:31:06.198831+0000 mon.a (mon.0) 2462 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]': finished 2024-02-18T01:31:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: cluster 2024-02-18T01:31:06.199086+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-18T01:31:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: audit 2024-02-18T01:31:06.200573+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[19734]: audit 2024-02-18T01:31:06.200649+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: audit 2024-02-18T01:31:05.203287+0000 mon.b (mon.1) 395 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: audit 2024-02-18T01:31:05.203716+0000 mon.a (mon.0) 2461 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]: dispatch 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: cluster 2024-02-18T01:31:05.874638+0000 mgr.y (mgr.24475) 552 : cluster [DBG] pgmap v801: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: audit 2024-02-18T01:31:06.198831+0000 mon.a (mon.0) 2462 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_tier","val": "test-rados-api-smithi105-61833-85-test-flush"}]': finished 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: cluster 2024-02-18T01:31:06.199086+0000 mon.a (mon.0) 2463 : cluster [DBG] osdmap e455: 8 total, 8 up, 8 in 2024-02-18T01:31:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: audit 2024-02-18T01:31:06.200573+0000 mon.b (mon.1) 396 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:06 smithi155 bash[20805]: audit 2024-02-18T01:31:06.200649+0000 mon.a (mon.0) 2464 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:06.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:06] "GET /metrics HTTP/1.1" 200 191963 "" "Prometheus/2.33.4" 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: audit 2024-02-18T01:31:07.203469+0000 mon.a (mon.0) 2465 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: cluster 2024-02-18T01:31:07.203817+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: audit 2024-02-18T01:31:07.206138+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: audit 2024-02-18T01:31:07.206139+0000 mon.a (mon.0) 2467 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: cluster 2024-02-18T01:31:07.877036+0000 mgr.y (mgr.24475) 553 : cluster [DBG] pgmap v804: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[19734]: audit 2024-02-18T01:31:07.999907+0000 mgr.y (mgr.24475) 554 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:08.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: audit 2024-02-18T01:31:07.203469+0000 mon.a (mon.0) 2465 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: cluster 2024-02-18T01:31:07.203817+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-18T01:31:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: audit 2024-02-18T01:31:07.206138+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: audit 2024-02-18T01:31:07.206139+0000 mon.a (mon.0) 2467 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: cluster 2024-02-18T01:31:07.877036+0000 mgr.y (mgr.24475) 553 : cluster [DBG] pgmap v804: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:08 smithi105 bash[27669]: audit 2024-02-18T01:31:07.999907+0000 mgr.y (mgr.24475) 554 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: audit 2024-02-18T01:31:07.203469+0000 mon.a (mon.0) 2465 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: cluster 2024-02-18T01:31:07.203817+0000 mon.a (mon.0) 2466 : cluster [DBG] osdmap e456: 8 total, 8 up, 8 in 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: audit 2024-02-18T01:31:07.206138+0000 mon.b (mon.1) 397 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: audit 2024-02-18T01:31:07.206139+0000 mon.a (mon.0) 2467 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: cluster 2024-02-18T01:31:07.877036+0000 mgr.y (mgr.24475) 553 : cluster [DBG] pgmap v804: 260 pgs: 260 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:08 smithi155 bash[20805]: audit 2024-02-18T01:31:07.999907+0000 mgr.y (mgr.24475) 554 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:09.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:09 smithi105 bash[27669]: audit 2024-02-18T01:31:08.203977+0000 mon.a (mon.0) 2468 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:09 smithi105 bash[27669]: cluster 2024-02-18T01:31:08.204176+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-18T01:31:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:09 smithi105 bash[19734]: audit 2024-02-18T01:31:08.203977+0000 mon.a (mon.0) 2468 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:09 smithi105 bash[19734]: cluster 2024-02-18T01:31:08.204176+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-18T01:31:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:09 smithi155 bash[20805]: audit 2024-02-18T01:31:08.203977+0000 mon.a (mon.0) 2468 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:09 smithi155 bash[20805]: cluster 2024-02-18T01:31:08.204176+0000 mon.a (mon.0) 2469 : cluster [DBG] osdmap e457: 8 total, 8 up, 8 in 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: cluster 2024-02-18T01:31:09.216581+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: audit 2024-02-18T01:31:09.260672+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: audit 2024-02-18T01:31:09.260824+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: audit 2024-02-18T01:31:09.261756+0000 mon.a (mon.0) 2472 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: audit 2024-02-18T01:31:09.262086+0000 mon.b (mon.1) 399 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[27669]: cluster 2024-02-18T01:31:09.877885+0000 mgr.y (mgr.24475) 555 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: cluster 2024-02-18T01:31:09.216581+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-18T01:31:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: audit 2024-02-18T01:31:09.260672+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: audit 2024-02-18T01:31:09.260824+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: audit 2024-02-18T01:31:09.261756+0000 mon.a (mon.0) 2472 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: audit 2024-02-18T01:31:09.262086+0000 mon.b (mon.1) 399 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:10 smithi105 bash[19734]: cluster 2024-02-18T01:31:09.877885+0000 mgr.y (mgr.24475) 555 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: cluster 2024-02-18T01:31:09.216581+0000 mon.a (mon.0) 2470 : cluster [DBG] osdmap e458: 8 total, 8 up, 8 in 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: audit 2024-02-18T01:31:09.260672+0000 mon.a (mon.0) 2471 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: audit 2024-02-18T01:31:09.260824+0000 mon.b (mon.1) 398 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: audit 2024-02-18T01:31:09.261756+0000 mon.a (mon.0) 2472 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: audit 2024-02-18T01:31:09.262086+0000 mon.b (mon.1) 399 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-83"}]: dispatch 2024-02-18T01:31:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:10 smithi155 bash[20805]: cluster 2024-02-18T01:31:09.877885+0000 mgr.y (mgr.24475) 555 : cluster [DBG] pgmap v807: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:11 smithi155 bash[20805]: cluster 2024-02-18T01:31:10.238900+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-18T01:31:11.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:11 smithi105 bash[19734]: cluster 2024-02-18T01:31:10.238900+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-18T01:31:11.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:11 smithi105 bash[27669]: cluster 2024-02-18T01:31:10.238900+0000 mon.a (mon.0) 2473 : cluster [DBG] osdmap e459: 8 total, 8 up, 8 in 2024-02-18T01:31:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[27669]: cluster 2024-02-18T01:31:11.259270+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[27669]: audit 2024-02-18T01:31:11.263058+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[27669]: audit 2024-02-18T01:31:11.267525+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[27669]: cluster 2024-02-18T01:31:11.878727+0000 mgr.y (mgr.24475) 556 : cluster [DBG] pgmap v810: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[19734]: cluster 2024-02-18T01:31:11.259270+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[19734]: audit 2024-02-18T01:31:11.263058+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[19734]: audit 2024-02-18T01:31:11.267525+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:12 smithi105 bash[19734]: cluster 2024-02-18T01:31:11.878727+0000 mgr.y (mgr.24475) 556 : cluster [DBG] pgmap v810: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:12 smithi155 bash[20805]: cluster 2024-02-18T01:31:11.259270+0000 mon.a (mon.0) 2474 : cluster [DBG] osdmap e460: 8 total, 8 up, 8 in 2024-02-18T01:31:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:12 smithi155 bash[20805]: audit 2024-02-18T01:31:11.263058+0000 mon.b (mon.1) 400 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:12 smithi155 bash[20805]: audit 2024-02-18T01:31:11.267525+0000 mon.a (mon.0) 2475 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:12 smithi155 bash[20805]: cluster 2024-02-18T01:31:11.878727+0000 mgr.y (mgr.24475) 556 : cluster [DBG] pgmap v810: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[19734]: audit 2024-02-18T01:31:12.269665+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[19734]: cluster 2024-02-18T01:31:12.269834+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[19734]: audit 2024-02-18T01:31:12.276991+0000 mon.b (mon.1) 401 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[19734]: audit 2024-02-18T01:31:12.280478+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[19734]: audit 2024-02-18T01:31:12.284107+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:13.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[27669]: audit 2024-02-18T01:31:12.269665+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[27669]: cluster 2024-02-18T01:31:12.269834+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-18T01:31:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[27669]: audit 2024-02-18T01:31:12.276991+0000 mon.b (mon.1) 401 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[27669]: audit 2024-02-18T01:31:12.280478+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:13.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:13 smithi105 bash[27669]: audit 2024-02-18T01:31:12.284107+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:13 smithi155 bash[20805]: audit 2024-02-18T01:31:12.269665+0000 mon.a (mon.0) 2476 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-86","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:13 smithi155 bash[20805]: cluster 2024-02-18T01:31:12.269834+0000 mon.a (mon.0) 2477 : cluster [DBG] osdmap e461: 8 total, 8 up, 8 in 2024-02-18T01:31:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:13 smithi155 bash[20805]: audit 2024-02-18T01:31:12.276991+0000 mon.b (mon.1) 401 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:13 smithi155 bash[20805]: audit 2024-02-18T01:31:12.280478+0000 mon.b (mon.1) 402 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:13.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:13 smithi155 bash[20805]: audit 2024-02-18T01:31:12.284107+0000 mon.a (mon.0) 2478 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[19734]: audit 2024-02-18T01:31:13.284126+0000 mon.a (mon.0) 2479 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[19734]: cluster 2024-02-18T01:31:13.284364+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[19734]: cluster 2024-02-18T01:31:13.879565+0000 mgr.y (mgr.24475) 557 : cluster [DBG] pgmap v813: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[27669]: audit 2024-02-18T01:31:13.284126+0000 mon.a (mon.0) 2479 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[27669]: cluster 2024-02-18T01:31:13.284364+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-18T01:31:14.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:14 smithi105 bash[27669]: cluster 2024-02-18T01:31:13.879565+0000 mgr.y (mgr.24475) 557 : cluster [DBG] pgmap v813: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:14.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:14 smithi155 bash[20805]: audit 2024-02-18T01:31:13.284126+0000 mon.a (mon.0) 2479 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:14.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:14 smithi155 bash[20805]: cluster 2024-02-18T01:31:13.284364+0000 mon.a (mon.0) 2480 : cluster [DBG] osdmap e462: 8 total, 8 up, 8 in 2024-02-18T01:31:14.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:14 smithi155 bash[20805]: cluster 2024-02-18T01:31:13.879565+0000 mgr.y (mgr.24475) 557 : cluster [DBG] pgmap v813: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:15.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:15 smithi105 bash[19734]: cluster 2024-02-18T01:31:14.296438+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-18T01:31:15.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:15 smithi105 bash[27669]: cluster 2024-02-18T01:31:14.296438+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-18T01:31:15.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:15 smithi155 bash[20805]: cluster 2024-02-18T01:31:14.296438+0000 mon.a (mon.0) 2481 : cluster [DBG] osdmap e463: 8 total, 8 up, 8 in 2024-02-18T01:31:16.316 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22159 ms) 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3022 ms) 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3041 ms) 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3037 ms) 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3031 ms) 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.317 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4070 ms) 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (38187 ms) 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (16955 ms) 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4067 ms) 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3038 ms) 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-02-18T01:31:16.318 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25163 ms) 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (14761 ms) 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5053 ms) 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: waiting for scrubs... 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: done waiting 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24977 ms) 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.319 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10065 ms) 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9116 ms) 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9061 ms) 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9056 ms) 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:31:16.320 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6075 ms) 2024-02-18T01:31:16.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: cluster 2024-02-18T01:31:15.294907+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T01:31:16.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: audit 2024-02-18T01:31:15.336060+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: audit 2024-02-18T01:31:15.336153+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: audit 2024-02-18T01:31:15.337215+0000 mon.a (mon.0) 2484 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: audit 2024-02-18T01:31:15.337430+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[27669]: cluster 2024-02-18T01:31:15.880514+0000 mgr.y (mgr.24475) 558 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: cluster 2024-02-18T01:31:15.294907+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: audit 2024-02-18T01:31:15.336060+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: audit 2024-02-18T01:31:15.336153+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: audit 2024-02-18T01:31:15.337215+0000 mon.a (mon.0) 2484 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: audit 2024-02-18T01:31:15.337430+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[19734]: cluster 2024-02-18T01:31:15.880514+0000 mgr.y (mgr.24475) 558 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: cluster 2024-02-18T01:31:15.294907+0000 mon.a (mon.0) 2482 : cluster [DBG] osdmap e464: 8 total, 8 up, 8 in 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: audit 2024-02-18T01:31:15.336060+0000 mon.a (mon.0) 2483 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: audit 2024-02-18T01:31:15.336153+0000 mon.b (mon.1) 403 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: audit 2024-02-18T01:31:15.337215+0000 mon.a (mon.0) 2484 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: audit 2024-02-18T01:31:15.337430+0000 mon.b (mon.1) 404 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-86"}]: dispatch 2024-02-18T01:31:16.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:16 smithi155 bash[20805]: cluster 2024-02-18T01:31:15.880514+0000 mgr.y (mgr.24475) 558 : cluster [DBG] pgmap v816: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:16.991 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:16] "GET /metrics HTTP/1.1" 200 191963 "" "Prometheus/2.33.4" 2024-02-18T01:31:17.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:17 smithi105 bash[19734]: cluster 2024-02-18T01:31:16.314112+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T01:31:17.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:17 smithi105 bash[27669]: cluster 2024-02-18T01:31:16.314112+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T01:31:17.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:17 smithi155 bash[20805]: cluster 2024-02-18T01:31:16.314112+0000 mon.a (mon.0) 2485 : cluster [DBG] osdmap e465: 8 total, 8 up, 8 in 2024-02-18T01:31:18.326 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[27669]: cluster 2024-02-18T01:31:17.327743+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[27669]: audit 2024-02-18T01:31:17.334079+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[27669]: audit 2024-02-18T01:31:17.334513+0000 mon.a (mon.0) 2487 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[27669]: cluster 2024-02-18T01:31:17.882920+0000 mgr.y (mgr.24475) 559 : cluster [DBG] pgmap v819: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[27669]: audit 2024-02-18T01:31:18.009690+0000 mgr.y (mgr.24475) 560 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[19734]: cluster 2024-02-18T01:31:17.327743+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[19734]: audit 2024-02-18T01:31:17.334079+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[19734]: audit 2024-02-18T01:31:17.334513+0000 mon.a (mon.0) 2487 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[19734]: cluster 2024-02-18T01:31:17.882920+0000 mgr.y (mgr.24475) 559 : cluster [DBG] pgmap v819: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:18.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:18 smithi105 bash[19734]: audit 2024-02-18T01:31:18.009690+0000 mgr.y (mgr.24475) 560 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[20805]: cluster 2024-02-18T01:31:17.327743+0000 mon.a (mon.0) 2486 : cluster [DBG] osdmap e466: 8 total, 8 up, 8 in 2024-02-18T01:31:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[20805]: audit 2024-02-18T01:31:17.334079+0000 mon.b (mon.1) 405 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[20805]: audit 2024-02-18T01:31:17.334513+0000 mon.a (mon.0) 2487 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[20805]: cluster 2024-02-18T01:31:17.882920+0000 mgr.y (mgr.24475) 559 : cluster [DBG] pgmap v819: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:18.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:18 smithi155 bash[20805]: audit 2024-02-18T01:31:18.009690+0000 mgr.y (mgr.24475) 560 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[19734]: audit 2024-02-18T01:31:18.339854+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[19734]: cluster 2024-02-18T01:31:18.340005+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T01:31:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[19734]: audit 2024-02-18T01:31:18.364523+0000 mon.a (mon.0) 2490 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[19734]: audit 2024-02-18T01:31:18.364699+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[27669]: audit 2024-02-18T01:31:18.339854+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[27669]: cluster 2024-02-18T01:31:18.340005+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T01:31:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[27669]: audit 2024-02-18T01:31:18.364523+0000 mon.a (mon.0) 2490 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:19 smithi105 bash[27669]: audit 2024-02-18T01:31:18.364699+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:19 smithi155 bash[20805]: audit 2024-02-18T01:31:18.339854+0000 mon.a (mon.0) 2488 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-88","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:19 smithi155 bash[20805]: cluster 2024-02-18T01:31:18.340005+0000 mon.a (mon.0) 2489 : cluster [DBG] osdmap e467: 8 total, 8 up, 8 in 2024-02-18T01:31:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:19 smithi155 bash[20805]: audit 2024-02-18T01:31:18.364523+0000 mon.a (mon.0) 2490 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:19 smithi155 bash[20805]: audit 2024-02-18T01:31:18.364699+0000 mon.b (mon.1) 406 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: audit 2024-02-18T01:31:19.347053+0000 mon.a (mon.0) 2491 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: cluster 2024-02-18T01:31:19.347260+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: audit 2024-02-18T01:31:19.348569+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: audit 2024-02-18T01:31:19.348662+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: cluster 2024-02-18T01:31:19.883823+0000 mgr.y (mgr.24475) 561 : cluster [DBG] pgmap v822: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:20.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[27669]: cluster 2024-02-18T01:31:20.339784+0000 mon.a (mon.0) 2494 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: audit 2024-02-18T01:31:19.347053+0000 mon.a (mon.0) 2491 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: cluster 2024-02-18T01:31:19.347260+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: audit 2024-02-18T01:31:19.348569+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: audit 2024-02-18T01:31:19.348662+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: cluster 2024-02-18T01:31:19.883823+0000 mgr.y (mgr.24475) 561 : cluster [DBG] pgmap v822: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:20.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:20 smithi105 bash[19734]: cluster 2024-02-18T01:31:20.339784+0000 mon.a (mon.0) 2494 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: audit 2024-02-18T01:31:19.347053+0000 mon.a (mon.0) 2491 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: cluster 2024-02-18T01:31:19.347260+0000 mon.a (mon.0) 2492 : cluster [DBG] osdmap e468: 8 total, 8 up, 8 in 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: audit 2024-02-18T01:31:19.348569+0000 mon.a (mon.0) 2493 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: audit 2024-02-18T01:31:19.348662+0000 mon.b (mon.1) 407 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]: dispatch 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: cluster 2024-02-18T01:31:19.883823+0000 mgr.y (mgr.24475) 561 : cluster [DBG] pgmap v822: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-02-18T01:31:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:20 smithi155 bash[20805]: cluster 2024-02-18T01:31:20.339784+0000 mon.a (mon.0) 2494 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: audit 2024-02-18T01:31:20.351406+0000 mon.a (mon.0) 2495 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]': finished 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: cluster 2024-02-18T01:31:20.351535+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: audit 2024-02-18T01:31:20.353073+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: audit 2024-02-18T01:31:20.353906+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: audit 2024-02-18T01:31:21.353515+0000 mon.a (mon.0) 2498 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[19734]: cluster 2024-02-18T01:31:21.353636+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: audit 2024-02-18T01:31:20.351406+0000 mon.a (mon.0) 2495 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]': finished 2024-02-18T01:31:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: cluster 2024-02-18T01:31:20.351535+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T01:31:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: audit 2024-02-18T01:31:20.353073+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: audit 2024-02-18T01:31:20.353906+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: audit 2024-02-18T01:31:21.353515+0000 mon.a (mon.0) 2498 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:21 smithi105 bash[27669]: cluster 2024-02-18T01:31:21.353636+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T01:31:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: audit 2024-02-18T01:31:20.351406+0000 mon.a (mon.0) 2495 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-88", "mode": "writeback"}]': finished 2024-02-18T01:31:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: cluster 2024-02-18T01:31:20.351535+0000 mon.a (mon.0) 2496 : cluster [DBG] osdmap e469: 8 total, 8 up, 8 in 2024-02-18T01:31:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: audit 2024-02-18T01:31:20.353073+0000 mon.b (mon.1) 408 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: audit 2024-02-18T01:31:20.353906+0000 mon.a (mon.0) 2497 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: audit 2024-02-18T01:31:21.353515+0000 mon.a (mon.0) 2498 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:21.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[20805]: cluster 2024-02-18T01:31:21.353636+0000 mon.a (mon.0) 2499 : cluster [DBG] osdmap e470: 8 total, 8 up, 8 in 2024-02-18T01:31:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:21.355462+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:21.355482+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:21.516106+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: cluster 2024-02-18T01:31:21.884550+0000 mgr.y (mgr.24475) 562 : cluster [DBG] pgmap v825: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: cluster 2024-02-18T01:31:22.345022+0000 mon.a (mon.0) 2502 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:22.354572+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: cluster 2024-02-18T01:31:22.354736+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:22.356826+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[19734]: audit 2024-02-18T01:31:22.356846+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:21.355462+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:21.355482+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:21.516106+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: cluster 2024-02-18T01:31:21.884550+0000 mgr.y (mgr.24475) 562 : cluster [DBG] pgmap v825: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: cluster 2024-02-18T01:31:22.345022+0000 mon.a (mon.0) 2502 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:22.354572+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:31:22.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: cluster 2024-02-18T01:31:22.354736+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T01:31:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:22.356826+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:22.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:22 smithi105 bash[27669]: audit 2024-02-18T01:31:22.356846+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:21.355462+0000 mon.b (mon.1) 409 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:21.355482+0000 mon.a (mon.0) 2500 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:31:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:21.516106+0000 mon.a (mon.0) 2501 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: cluster 2024-02-18T01:31:21.884550+0000 mgr.y (mgr.24475) 562 : cluster [DBG] pgmap v825: 228 pgs: 32 creating+peering, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: cluster 2024-02-18T01:31:22.345022+0000 mon.a (mon.0) 2502 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:22.354572+0000 mon.a (mon.0) 2503 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:31:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: cluster 2024-02-18T01:31:22.354736+0000 mon.a (mon.0) 2504 : cluster [DBG] osdmap e471: 8 total, 8 up, 8 in 2024-02-18T01:31:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:22.356826+0000 mon.a (mon.0) 2505 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:22.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:22 smithi155 bash[20805]: audit 2024-02-18T01:31:22.356846+0000 mon.b (mon.1) 410 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]: dispatch 2024-02-18T01:31:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[19734]: audit 2024-02-18T01:31:23.356782+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T01:31:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[19734]: cluster 2024-02-18T01:31:23.356957+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T01:31:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[19734]: audit 2024-02-18T01:31:23.358490+0000 mon.a (mon.0) 2508 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[19734]: audit 2024-02-18T01:31:23.358497+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[19734]: cluster 2024-02-18T01:31:23.885418+0000 mgr.y (mgr.24475) 563 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[27669]: audit 2024-02-18T01:31:23.356782+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T01:31:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[27669]: cluster 2024-02-18T01:31:23.356957+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T01:31:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[27669]: audit 2024-02-18T01:31:23.358490+0000 mon.a (mon.0) 2508 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[27669]: audit 2024-02-18T01:31:23.358497+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.742 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:24 smithi105 bash[27669]: cluster 2024-02-18T01:31:23.885418+0000 mgr.y (mgr.24475) 563 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:24 smithi155 bash[20805]: audit 2024-02-18T01:31:23.356782+0000 mon.a (mon.0) 2506 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_count","val": "1"}]': finished 2024-02-18T01:31:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:24 smithi155 bash[20805]: cluster 2024-02-18T01:31:23.356957+0000 mon.a (mon.0) 2507 : cluster [DBG] osdmap e472: 8 total, 8 up, 8 in 2024-02-18T01:31:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:24 smithi155 bash[20805]: audit 2024-02-18T01:31:23.358490+0000 mon.a (mon.0) 2508 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:24 smithi155 bash[20805]: audit 2024-02-18T01:31:23.358497+0000 mon.b (mon.1) 411 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:24.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:24 smithi155 bash[20805]: cluster 2024-02-18T01:31:23.885418+0000 mgr.y (mgr.24475) 563 : cluster [DBG] pgmap v828: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[19734]: audit 2024-02-18T01:31:24.357803+0000 mon.a (mon.0) 2509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[19734]: cluster 2024-02-18T01:31:24.357911+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[19734]: audit 2024-02-18T01:31:24.361131+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[19734]: audit 2024-02-18T01:31:24.366134+0000 mon.a (mon.0) 2511 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[27669]: audit 2024-02-18T01:31:24.357803+0000 mon.a (mon.0) 2509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[27669]: cluster 2024-02-18T01:31:24.357911+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[27669]: audit 2024-02-18T01:31:24.361131+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:25 smithi105 bash[27669]: audit 2024-02-18T01:31:24.366134+0000 mon.a (mon.0) 2511 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:25 smithi155 bash[20805]: audit 2024-02-18T01:31:24.357803+0000 mon.a (mon.0) 2509 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:25 smithi155 bash[20805]: cluster 2024-02-18T01:31:24.357911+0000 mon.a (mon.0) 2510 : cluster [DBG] osdmap e473: 8 total, 8 up, 8 in 2024-02-18T01:31:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:25 smithi155 bash[20805]: audit 2024-02-18T01:31:24.361131+0000 mon.b (mon.1) 412 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:25 smithi155 bash[20805]: audit 2024-02-18T01:31:24.366134+0000 mon.a (mon.0) 2511 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]: dispatch 2024-02-18T01:31:26.678 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: audit 2024-02-18T01:31:25.369989+0000 mon.a (mon.0) 2512 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: cluster 2024-02-18T01:31:25.370172+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: audit 2024-02-18T01:31:25.419708+0000 mon.a (mon.0) 2514 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: audit 2024-02-18T01:31:25.419717+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: audit 2024-02-18T01:31:25.797804+0000 mon.a (mon.0) 2515 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: cluster 2024-02-18T01:31:25.886243+0000 mgr.y (mgr.24475) 564 : cluster [DBG] pgmap v831: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[19734]: audit 2024-02-18T01:31:26.108891+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: audit 2024-02-18T01:31:25.369989+0000 mon.a (mon.0) 2512 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T01:31:26.679 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: cluster 2024-02-18T01:31:25.370172+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T01:31:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: audit 2024-02-18T01:31:25.419708+0000 mon.a (mon.0) 2514 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: audit 2024-02-18T01:31:25.419717+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: audit 2024-02-18T01:31:25.797804+0000 mon.a (mon.0) 2515 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:31:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: cluster 2024-02-18T01:31:25.886243+0000 mgr.y (mgr.24475) 564 : cluster [DBG] pgmap v831: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:26.680 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[27669]: audit 2024-02-18T01:31:26.108891+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: audit 2024-02-18T01:31:25.369989+0000 mon.a (mon.0) 2512 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-88","var": "target_max_objects","val": "250"}]': finished 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: cluster 2024-02-18T01:31:25.370172+0000 mon.a (mon.0) 2513 : cluster [DBG] osdmap e474: 8 total, 8 up, 8 in 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: audit 2024-02-18T01:31:25.419708+0000 mon.a (mon.0) 2514 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: audit 2024-02-18T01:31:25.419717+0000 mon.b (mon.1) 413 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: audit 2024-02-18T01:31:25.797804+0000 mon.a (mon.0) 2515 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: cluster 2024-02-18T01:31:25.886243+0000 mgr.y (mgr.24475) 564 : cluster [DBG] pgmap v831: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:31:26.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:26 smithi155 bash[20805]: audit 2024-02-18T01:31:26.108891+0000 mon.a (mon.0) 2516 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:26] "GET /metrics HTTP/1.1" 200 189667 "" "Prometheus/2.33.4" 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.388819+0000 mon.a (mon.0) 2517 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: cluster 2024-02-18T01:31:26.388888+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.390527+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.393274+0000 mon.a (mon.0) 2519 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.423553+0000 mon.a (mon.0) 2520 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.425055+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[19734]: audit 2024-02-18T01:31:26.433707+0000 mon.a (mon.0) 2522 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.388819+0000 mon.a (mon.0) 2517 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:27.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: cluster 2024-02-18T01:31:26.388888+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T01:31:27.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.390527+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.393274+0000 mon.a (mon.0) 2519 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.423553+0000 mon.a (mon.0) 2520 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:31:27.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.425055+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:31:27.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:27 smithi105 bash[27669]: audit 2024-02-18T01:31:26.433707+0000 mon.a (mon.0) 2522 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.388819+0000 mon.a (mon.0) 2517 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: cluster 2024-02-18T01:31:26.388888+0000 mon.a (mon.0) 2518 : cluster [DBG] osdmap e475: 8 total, 8 up, 8 in 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.390527+0000 mon.b (mon.1) 414 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.393274+0000 mon.a (mon.0) 2519 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]: dispatch 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.423553+0000 mon.a (mon.0) 2520 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.425055+0000 mon.a (mon.0) 2521 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:31:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:27 smithi155 bash[20805]: audit 2024-02-18T01:31:26.433707+0000 mon.a (mon.0) 2522 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:28.400 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:28.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:28 smithi155 bash[20805]: audit 2024-02-18T01:31:27.391847+0000 mon.a (mon.0) 2523 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:28.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:28 smithi155 bash[20805]: cluster 2024-02-18T01:31:27.391965+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T01:31:28.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:28 smithi155 bash[20805]: cluster 2024-02-18T01:31:27.887320+0000 mgr.y (mgr.24475) 565 : cluster [DBG] pgmap v834: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:28.401 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:28 smithi155 bash[20805]: audit 2024-02-18T01:31:28.020999+0000 mgr.y (mgr.24475) 566 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[27669]: audit 2024-02-18T01:31:27.391847+0000 mon.a (mon.0) 2523 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[27669]: cluster 2024-02-18T01:31:27.391965+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[27669]: cluster 2024-02-18T01:31:27.887320+0000 mgr.y (mgr.24475) 565 : cluster [DBG] pgmap v834: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[27669]: audit 2024-02-18T01:31:28.020999+0000 mgr.y (mgr.24475) 566 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[19734]: audit 2024-02-18T01:31:27.391847+0000 mon.a (mon.0) 2523 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-88"}]': finished 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[19734]: cluster 2024-02-18T01:31:27.391965+0000 mon.a (mon.0) 2524 : cluster [DBG] osdmap e476: 8 total, 8 up, 8 in 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[19734]: cluster 2024-02-18T01:31:27.887320+0000 mgr.y (mgr.24475) 565 : cluster [DBG] pgmap v834: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:28 smithi105 bash[19734]: audit 2024-02-18T01:31:28.020999+0000 mgr.y (mgr.24475) 566 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:29 smithi105 bash[19734]: cluster 2024-02-18T01:31:28.414609+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T01:31:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:29 smithi105 bash[27669]: cluster 2024-02-18T01:31:28.414609+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T01:31:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:29 smithi155 bash[20805]: cluster 2024-02-18T01:31:28.414609+0000 mon.a (mon.0) 2525 : cluster [DBG] osdmap e477: 8 total, 8 up, 8 in 2024-02-18T01:31:30.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[27669]: cluster 2024-02-18T01:31:29.430122+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[27669]: audit 2024-02-18T01:31:29.435496+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[27669]: audit 2024-02-18T01:31:29.435840+0000 mon.a (mon.0) 2527 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[27669]: cluster 2024-02-18T01:31:29.888121+0000 mgr.y (mgr.24475) 567 : cluster [DBG] pgmap v837: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[19734]: cluster 2024-02-18T01:31:29.430122+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[19734]: audit 2024-02-18T01:31:29.435496+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[19734]: audit 2024-02-18T01:31:29.435840+0000 mon.a (mon.0) 2527 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:30 smithi105 bash[19734]: cluster 2024-02-18T01:31:29.888121+0000 mgr.y (mgr.24475) 567 : cluster [DBG] pgmap v837: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:30 smithi155 bash[20805]: cluster 2024-02-18T01:31:29.430122+0000 mon.a (mon.0) 2526 : cluster [DBG] osdmap e478: 8 total, 8 up, 8 in 2024-02-18T01:31:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:30 smithi155 bash[20805]: audit 2024-02-18T01:31:29.435496+0000 mon.b (mon.1) 415 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:30 smithi155 bash[20805]: audit 2024-02-18T01:31:29.435840+0000 mon.a (mon.0) 2527 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:30 smithi155 bash[20805]: cluster 2024-02-18T01:31:29.888121+0000 mgr.y (mgr.24475) 567 : cluster [DBG] pgmap v837: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[19734]: audit 2024-02-18T01:31:30.434275+0000 mon.a (mon.0) 2528 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[19734]: cluster 2024-02-18T01:31:30.434477+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[19734]: audit 2024-02-18T01:31:30.449394+0000 mon.a (mon.0) 2530 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[19734]: audit 2024-02-18T01:31:30.449446+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[27669]: audit 2024-02-18T01:31:30.434275+0000 mon.a (mon.0) 2528 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[27669]: cluster 2024-02-18T01:31:30.434477+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T01:31:31.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[27669]: audit 2024-02-18T01:31:30.449394+0000 mon.a (mon.0) 2530 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:31.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:31 smithi105 bash[27669]: audit 2024-02-18T01:31:30.449446+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:31.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:31 smithi155 bash[20805]: audit 2024-02-18T01:31:30.434275+0000 mon.a (mon.0) 2528 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:31.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:31 smithi155 bash[20805]: cluster 2024-02-18T01:31:30.434477+0000 mon.a (mon.0) 2529 : cluster [DBG] osdmap e479: 8 total, 8 up, 8 in 2024-02-18T01:31:31.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:31 smithi155 bash[20805]: audit 2024-02-18T01:31:30.449394+0000 mon.a (mon.0) 2530 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:31.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:31 smithi155 bash[20805]: audit 2024-02-18T01:31:30.449446+0000 mon.b (mon.1) 416 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:31:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[19734]: audit 2024-02-18T01:31:31.459121+0000 mon.a (mon.0) 2531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[19734]: cluster 2024-02-18T01:31:31.459243+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[19734]: audit 2024-02-18T01:31:31.462555+0000 mon.a (mon.0) 2533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[19734]: audit 2024-02-18T01:31:31.462656+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[19734]: cluster 2024-02-18T01:31:31.888938+0000 mgr.y (mgr.24475) 568 : cluster [DBG] pgmap v840: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:32.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[27669]: audit 2024-02-18T01:31:31.459121+0000 mon.a (mon.0) 2531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[27669]: cluster 2024-02-18T01:31:31.459243+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T01:31:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[27669]: audit 2024-02-18T01:31:31.462555+0000 mon.a (mon.0) 2533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[27669]: audit 2024-02-18T01:31:31.462656+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:32 smithi105 bash[27669]: cluster 2024-02-18T01:31:31.888938+0000 mgr.y (mgr.24475) 568 : cluster [DBG] pgmap v840: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:32 smithi155 bash[20805]: audit 2024-02-18T01:31:31.459121+0000 mon.a (mon.0) 2531 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:31:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:32 smithi155 bash[20805]: cluster 2024-02-18T01:31:31.459243+0000 mon.a (mon.0) 2532 : cluster [DBG] osdmap e480: 8 total, 8 up, 8 in 2024-02-18T01:31:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:32 smithi155 bash[20805]: audit 2024-02-18T01:31:31.462555+0000 mon.a (mon.0) 2533 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:32 smithi155 bash[20805]: audit 2024-02-18T01:31:31.462656+0000 mon.b (mon.1) 417 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:32.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:32 smithi155 bash[20805]: cluster 2024-02-18T01:31:31.888938+0000 mgr.y (mgr.24475) 568 : cluster [DBG] pgmap v840: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:33.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[19734]: audit 2024-02-18T01:31:32.472156+0000 mon.a (mon.0) 2534 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[19734]: cluster 2024-02-18T01:31:32.472261+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[19734]: audit 2024-02-18T01:31:32.473794+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[19734]: audit 2024-02-18T01:31:32.474142+0000 mon.a (mon.0) 2536 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[27669]: audit 2024-02-18T01:31:32.472156+0000 mon.a (mon.0) 2534 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[27669]: cluster 2024-02-18T01:31:32.472261+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[27669]: audit 2024-02-18T01:31:32.473794+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:33.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:33 smithi105 bash[27669]: audit 2024-02-18T01:31:32.474142+0000 mon.a (mon.0) 2536 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:33 smithi155 bash[20805]: audit 2024-02-18T01:31:32.472156+0000 mon.a (mon.0) 2534 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-6", "overlaypool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:33 smithi155 bash[20805]: cluster 2024-02-18T01:31:32.472261+0000 mon.a (mon.0) 2535 : cluster [DBG] osdmap e481: 8 total, 8 up, 8 in 2024-02-18T01:31:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:33 smithi155 bash[20805]: audit 2024-02-18T01:31:32.473794+0000 mon.b (mon.1) 418 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:33.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:33 smithi155 bash[20805]: audit 2024-02-18T01:31:32.474142+0000 mon.a (mon.0) 2536 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]: dispatch 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: cluster 2024-02-18T01:31:33.466852+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: audit 2024-02-18T01:31:33.476039+0000 mon.a (mon.0) 2538 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]': finished 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: cluster 2024-02-18T01:31:33.476219+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: audit 2024-02-18T01:31:33.478855+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: audit 2024-02-18T01:31:33.484262+0000 mon.a (mon.0) 2540 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[27669]: cluster 2024-02-18T01:31:33.889806+0000 mgr.y (mgr.24475) 569 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:34.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: cluster 2024-02-18T01:31:33.466852+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:34.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: audit 2024-02-18T01:31:33.476039+0000 mon.a (mon.0) 2538 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]': finished 2024-02-18T01:31:34.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: cluster 2024-02-18T01:31:33.476219+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T01:31:34.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: audit 2024-02-18T01:31:33.478855+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: audit 2024-02-18T01:31:33.484262+0000 mon.a (mon.0) 2540 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.741 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:34 smithi105 bash[19734]: cluster 2024-02-18T01:31:33.889806+0000 mgr.y (mgr.24475) 569 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:34.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: cluster 2024-02-18T01:31:33.466852+0000 mon.a (mon.0) 2537 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:34.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: audit 2024-02-18T01:31:33.476039+0000 mon.a (mon.0) 2538 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-90", "mode": "writeback"}]': finished 2024-02-18T01:31:34.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: cluster 2024-02-18T01:31:33.476219+0000 mon.a (mon.0) 2539 : cluster [DBG] osdmap e482: 8 total, 8 up, 8 in 2024-02-18T01:31:34.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: audit 2024-02-18T01:31:33.478855+0000 mon.b (mon.1) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: audit 2024-02-18T01:31:33.484262+0000 mon.a (mon.0) 2540 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:31:34.744 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:34 smithi155 bash[20805]: cluster 2024-02-18T01:31:33.889806+0000 mgr.y (mgr.24475) 569 : cluster [DBG] pgmap v843: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[27669]: audit 2024-02-18T01:31:34.495880+0000 mon.a (mon.0) 2541 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[27669]: cluster 2024-02-18T01:31:34.496019+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[27669]: audit 2024-02-18T01:31:34.499391+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[27669]: audit 2024-02-18T01:31:34.499714+0000 mon.a (mon.0) 2543 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[19734]: audit 2024-02-18T01:31:34.495880+0000 mon.a (mon.0) 2541 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[19734]: cluster 2024-02-18T01:31:34.496019+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[19734]: audit 2024-02-18T01:31:34.499391+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:35 smithi105 bash[19734]: audit 2024-02-18T01:31:34.499714+0000 mon.a (mon.0) 2543 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:35 smithi155 bash[20805]: audit 2024-02-18T01:31:34.495880+0000 mon.a (mon.0) 2541 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:31:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:35 smithi155 bash[20805]: cluster 2024-02-18T01:31:34.496019+0000 mon.a (mon.0) 2542 : cluster [DBG] osdmap e483: 8 total, 8 up, 8 in 2024-02-18T01:31:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:35 smithi155 bash[20805]: audit 2024-02-18T01:31:34.499391+0000 mon.b (mon.1) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:35 smithi155 bash[20805]: audit 2024-02-18T01:31:34.499714+0000 mon.a (mon.0) 2543 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:31:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:36] "GET /metrics HTTP/1.1" 200 189785 "" "Prometheus/2.33.4" 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[27669]: audit 2024-02-18T01:31:35.501768+0000 mon.a (mon.0) 2544 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[27669]: cluster 2024-02-18T01:31:35.501975+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[27669]: audit 2024-02-18T01:31:35.505817+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[27669]: audit 2024-02-18T01:31:35.511795+0000 mon.a (mon.0) 2546 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[27669]: cluster 2024-02-18T01:31:35.890566+0000 mgr.y (mgr.24475) 570 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[19734]: audit 2024-02-18T01:31:35.501768+0000 mon.a (mon.0) 2544 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[19734]: cluster 2024-02-18T01:31:35.501975+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T01:31:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[19734]: audit 2024-02-18T01:31:35.505817+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[19734]: audit 2024-02-18T01:31:35.511795+0000 mon.a (mon.0) 2546 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:36 smithi105 bash[19734]: cluster 2024-02-18T01:31:35.890566+0000 mgr.y (mgr.24475) 570 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:36 smithi155 bash[20805]: audit 2024-02-18T01:31:35.501768+0000 mon.a (mon.0) 2544 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:31:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:36 smithi155 bash[20805]: cluster 2024-02-18T01:31:35.501975+0000 mon.a (mon.0) 2545 : cluster [DBG] osdmap e484: 8 total, 8 up, 8 in 2024-02-18T01:31:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:36 smithi155 bash[20805]: audit 2024-02-18T01:31:35.505817+0000 mon.b (mon.1) 421 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:36 smithi155 bash[20805]: audit 2024-02-18T01:31:35.511795+0000 mon.a (mon.0) 2546 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:31:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:36 smithi155 bash[20805]: cluster 2024-02-18T01:31:35.890566+0000 mgr.y (mgr.24475) 570 : cluster [DBG] pgmap v846: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: cluster 2024-02-18T01:31:36.507225+0000 mon.a (mon.0) 2547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: audit 2024-02-18T01:31:36.524529+0000 mon.a (mon.0) 2548 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: cluster 2024-02-18T01:31:36.524665+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: audit 2024-02-18T01:31:36.527257+0000 mon.a (mon.0) 2550 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: audit 2024-02-18T01:31:36.527385+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[19734]: audit 2024-02-18T01:31:36.861873+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]: dispatch 2024-02-18T01:31:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: cluster 2024-02-18T01:31:36.507225+0000 mon.a (mon.0) 2547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:37.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: audit 2024-02-18T01:31:36.524529+0000 mon.a (mon.0) 2548 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:31:37.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: cluster 2024-02-18T01:31:36.524665+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T01:31:37.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: audit 2024-02-18T01:31:36.527257+0000 mon.a (mon.0) 2550 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: audit 2024-02-18T01:31:36.527385+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:37 smithi105 bash[27669]: audit 2024-02-18T01:31:36.861873+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]: dispatch 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: cluster 2024-02-18T01:31:36.507225+0000 mon.a (mon.0) 2547 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: audit 2024-02-18T01:31:36.524529+0000 mon.a (mon.0) 2548 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: cluster 2024-02-18T01:31:36.524665+0000 mon.a (mon.0) 2549 : cluster [DBG] osdmap e485: 8 total, 8 up, 8 in 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: audit 2024-02-18T01:31:36.527257+0000 mon.a (mon.0) 2550 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: audit 2024-02-18T01:31:36.527385+0000 mon.b (mon.1) 422 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-02-18T01:31:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:37 smithi155 bash[20805]: audit 2024-02-18T01:31:36.861873+0000 mon.a (mon.0) 2551 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]: dispatch 2024-02-18T01:31:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[27669]: audit 2024-02-18T01:31:37.524132+0000 mon.a (mon.0) 2552 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[27669]: audit 2024-02-18T01:31:37.524245+0000 mon.a (mon.0) 2553 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]': finished 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[27669]: cluster 2024-02-18T01:31:37.524342+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[27669]: cluster 2024-02-18T01:31:37.891415+0000 mgr.y (mgr.24475) 571 : cluster [DBG] pgmap v849: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[27669]: audit 2024-02-18T01:31:38.032331+0000 mgr.y (mgr.24475) 572 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[19734]: audit 2024-02-18T01:31:37.524132+0000 mon.a (mon.0) 2552 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[19734]: audit 2024-02-18T01:31:37.524245+0000 mon.a (mon.0) 2553 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]': finished 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[19734]: cluster 2024-02-18T01:31:37.524342+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[19734]: cluster 2024-02-18T01:31:37.891415+0000 mgr.y (mgr.24475) 571 : cluster [DBG] pgmap v849: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:38 smithi105 bash[19734]: audit 2024-02-18T01:31:38.032331+0000 mgr.y (mgr.24475) 572 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[20805]: audit 2024-02-18T01:31:37.524132+0000 mon.a (mon.0) 2552 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-90","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-02-18T01:31:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[20805]: audit 2024-02-18T01:31:37.524245+0000 mon.a (mon.0) 2553 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "227.1", "id": [6, 1]}]': finished 2024-02-18T01:31:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[20805]: cluster 2024-02-18T01:31:37.524342+0000 mon.a (mon.0) 2554 : cluster [DBG] osdmap e486: 8 total, 8 up, 8 in 2024-02-18T01:31:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[20805]: cluster 2024-02-18T01:31:37.891415+0000 mgr.y (mgr.24475) 571 : cluster [DBG] pgmap v849: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:38 smithi155 bash[20805]: audit 2024-02-18T01:31:38.032331+0000 mgr.y (mgr.24475) 572 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[19734]: cluster 2024-02-18T01:31:38.539858+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T01:31:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[19734]: audit 2024-02-18T01:31:38.570411+0000 mon.a (mon.0) 2556 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[19734]: audit 2024-02-18T01:31:38.570593+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[27669]: cluster 2024-02-18T01:31:38.539858+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T01:31:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[27669]: audit 2024-02-18T01:31:38.570411+0000 mon.a (mon.0) 2556 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:39 smithi105 bash[27669]: audit 2024-02-18T01:31:38.570593+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:39 smithi155 bash[20805]: cluster 2024-02-18T01:31:38.539858+0000 mon.a (mon.0) 2555 : cluster [DBG] osdmap e487: 8 total, 8 up, 8 in 2024-02-18T01:31:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:39 smithi155 bash[20805]: audit 2024-02-18T01:31:38.570411+0000 mon.a (mon.0) 2556 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:39 smithi155 bash[20805]: audit 2024-02-18T01:31:38.570593+0000 mon.b (mon.1) 423 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[27669]: audit 2024-02-18T01:31:39.540123+0000 mon.a (mon.0) 2557 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[27669]: cluster 2024-02-18T01:31:39.540282+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[27669]: audit 2024-02-18T01:31:39.544526+0000 mon.b (mon.1) 424 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[27669]: audit 2024-02-18T01:31:39.548234+0000 mon.a (mon.0) 2559 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[27669]: cluster 2024-02-18T01:31:39.892215+0000 mgr.y (mgr.24475) 573 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[19734]: audit 2024-02-18T01:31:39.540123+0000 mon.a (mon.0) 2557 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[19734]: cluster 2024-02-18T01:31:39.540282+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T01:31:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[19734]: audit 2024-02-18T01:31:39.544526+0000 mon.b (mon.1) 424 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[19734]: audit 2024-02-18T01:31:39.548234+0000 mon.a (mon.0) 2559 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:40 smithi105 bash[19734]: cluster 2024-02-18T01:31:39.892215+0000 mgr.y (mgr.24475) 573 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:40 smithi155 bash[20805]: audit 2024-02-18T01:31:39.540123+0000 mon.a (mon.0) 2557 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]': finished 2024-02-18T01:31:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:40 smithi155 bash[20805]: cluster 2024-02-18T01:31:39.540282+0000 mon.a (mon.0) 2558 : cluster [DBG] osdmap e488: 8 total, 8 up, 8 in 2024-02-18T01:31:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:40 smithi155 bash[20805]: audit 2024-02-18T01:31:39.544526+0000 mon.b (mon.1) 424 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:40 smithi155 bash[20805]: audit 2024-02-18T01:31:39.548234+0000 mon.a (mon.0) 2559 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]: dispatch 2024-02-18T01:31:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:40 smithi155 bash[20805]: cluster 2024-02-18T01:31:39.892215+0000 mgr.y (mgr.24475) 573 : cluster [DBG] pgmap v852: 228 pgs: 228 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:42 smithi155 bash[20805]: audit 2024-02-18T01:31:40.566330+0000 mon.a (mon.0) 2560 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:42 smithi155 bash[20805]: cluster 2024-02-18T01:31:40.566474+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T01:31:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:42.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:42 smithi105 bash[27669]: audit 2024-02-18T01:31:40.566330+0000 mon.a (mon.0) 2560 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:42 smithi105 bash[27669]: cluster 2024-02-18T01:31:40.566474+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T01:31:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:42 smithi105 bash[19734]: audit 2024-02-18T01:31:40.566330+0000 mon.a (mon.0) 2560 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-90"}]': finished 2024-02-18T01:31:42.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:42 smithi105 bash[19734]: cluster 2024-02-18T01:31:40.566474+0000 mon.a (mon.0) 2561 : cluster [DBG] osdmap e489: 8 total, 8 up, 8 in 2024-02-18T01:31:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[19734]: cluster 2024-02-18T01:31:41.568046+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T01:31:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[19734]: cluster 2024-02-18T01:31:41.892937+0000 mgr.y (mgr.24475) 574 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[19734]: cluster 2024-02-18T01:31:42.575886+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T01:31:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[19734]: audit 2024-02-18T01:31:42.587938+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[19734]: audit 2024-02-18T01:31:42.588047+0000 mon.a (mon.0) 2564 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[27669]: cluster 2024-02-18T01:31:41.568046+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T01:31:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[27669]: cluster 2024-02-18T01:31:41.892937+0000 mgr.y (mgr.24475) 574 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[27669]: cluster 2024-02-18T01:31:42.575886+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T01:31:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[27669]: audit 2024-02-18T01:31:42.587938+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:43.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:43 smithi105 bash[27669]: audit 2024-02-18T01:31:42.588047+0000 mon.a (mon.0) 2564 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:43 smithi155 bash[20805]: cluster 2024-02-18T01:31:41.568046+0000 mon.a (mon.0) 2562 : cluster [DBG] osdmap e490: 8 total, 8 up, 8 in 2024-02-18T01:31:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:43 smithi155 bash[20805]: cluster 2024-02-18T01:31:41.892937+0000 mgr.y (mgr.24475) 574 : cluster [DBG] pgmap v855: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:43 smithi155 bash[20805]: cluster 2024-02-18T01:31:42.575886+0000 mon.a (mon.0) 2563 : cluster [DBG] osdmap e491: 8 total, 8 up, 8 in 2024-02-18T01:31:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:43 smithi155 bash[20805]: audit 2024-02-18T01:31:42.587938+0000 mon.b (mon.1) 425 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:43 smithi155 bash[20805]: audit 2024-02-18T01:31:42.588047+0000 mon.a (mon.0) 2564 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: audit 2024-02-18T01:31:43.579447+0000 mon.a (mon.0) 2565 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: cluster 2024-02-18T01:31:43.579914+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: audit 2024-02-18T01:31:43.582848+0000 mon.b (mon.1) 426 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: audit 2024-02-18T01:31:43.586678+0000 mon.a (mon.0) 2567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: audit 2024-02-18T01:31:43.586712+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[19734]: cluster 2024-02-18T01:31:43.893803+0000 mgr.y (mgr.24475) 575 : cluster [DBG] pgmap v858: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: audit 2024-02-18T01:31:43.579447+0000 mon.a (mon.0) 2565 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: cluster 2024-02-18T01:31:43.579914+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T01:31:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: audit 2024-02-18T01:31:43.582848+0000 mon.b (mon.1) 426 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:44.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: audit 2024-02-18T01:31:43.586678+0000 mon.a (mon.0) 2567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: audit 2024-02-18T01:31:43.586712+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:44 smithi105 bash[27669]: cluster 2024-02-18T01:31:43.893803+0000 mgr.y (mgr.24475) 575 : cluster [DBG] pgmap v858: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: audit 2024-02-18T01:31:43.579447+0000 mon.a (mon.0) 2565 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: cluster 2024-02-18T01:31:43.579914+0000 mon.a (mon.0) 2566 : cluster [DBG] osdmap e492: 8 total, 8 up, 8 in 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: audit 2024-02-18T01:31:43.582848+0000 mon.b (mon.1) 426 : audit [DBG] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: audit 2024-02-18T01:31:43.586678+0000 mon.a (mon.0) 2567 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: audit 2024-02-18T01:31:43.586712+0000 mon.b (mon.1) 427 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-02-18T01:31:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:44 smithi155 bash[20805]: cluster 2024-02-18T01:31:43.893803+0000 mgr.y (mgr.24475) 575 : cluster [DBG] pgmap v858: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:45.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[19734]: audit 2024-02-18T01:31:44.587634+0000 mon.a (mon.0) 2568 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[19734]: cluster 2024-02-18T01:31:44.587718+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[19734]: audit 2024-02-18T01:31:44.590627+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[19734]: audit 2024-02-18T01:31:44.590669+0000 mon.a (mon.0) 2570 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[27669]: audit 2024-02-18T01:31:44.587634+0000 mon.a (mon.0) 2568 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[27669]: cluster 2024-02-18T01:31:44.587718+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[27669]: audit 2024-02-18T01:31:44.590627+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:45.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:45 smithi105 bash[27669]: audit 2024-02-18T01:31:44.590669+0000 mon.a (mon.0) 2570 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:45 smithi155 bash[20805]: audit 2024-02-18T01:31:44.587634+0000 mon.a (mon.0) 2568 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-02-18T01:31:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:45 smithi155 bash[20805]: cluster 2024-02-18T01:31:44.587718+0000 mon.a (mon.0) 2569 : cluster [DBG] osdmap e493: 8 total, 8 up, 8 in 2024-02-18T01:31:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:45 smithi155 bash[20805]: audit 2024-02-18T01:31:44.590627+0000 mon.b (mon.1) 428 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:45.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:45 smithi155 bash[20805]: audit 2024-02-18T01:31:44.590669+0000 mon.a (mon.0) 2570 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-02-18T01:31:46.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:46] "GET /metrics HTTP/1.1" 200 189785 "" "Prometheus/2.33.4" 2024-02-18T01:31:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: audit 2024-02-18T01:31:45.592836+0000 mon.a (mon.0) 2571 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: cluster 2024-02-18T01:31:45.593069+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T01:31:46.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: audit 2024-02-18T01:31:45.594827+0000 mon.a (mon.0) 2573 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: audit 2024-02-18T01:31:45.594838+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: cluster 2024-02-18T01:31:45.894628+0000 mgr.y (mgr.24475) 576 : cluster [DBG] pgmap v861: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: audit 2024-02-18T01:31:46.594257+0000 mon.a (mon.0) 2574 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[27669]: cluster 2024-02-18T01:31:46.594472+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: audit 2024-02-18T01:31:45.592836+0000 mon.a (mon.0) 2571 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: cluster 2024-02-18T01:31:45.593069+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T01:31:46.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: audit 2024-02-18T01:31:45.594827+0000 mon.a (mon.0) 2573 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: audit 2024-02-18T01:31:45.594838+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: cluster 2024-02-18T01:31:45.894628+0000 mgr.y (mgr.24475) 576 : cluster [DBG] pgmap v861: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: audit 2024-02-18T01:31:46.594257+0000 mon.a (mon.0) 2574 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:46 smithi105 bash[19734]: cluster 2024-02-18T01:31:46.594472+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: audit 2024-02-18T01:31:45.592836+0000 mon.a (mon.0) 2571 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-02-18T01:31:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: cluster 2024-02-18T01:31:45.593069+0000 mon.a (mon.0) 2572 : cluster [DBG] osdmap e494: 8 total, 8 up, 8 in 2024-02-18T01:31:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: audit 2024-02-18T01:31:45.594827+0000 mon.a (mon.0) 2573 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: audit 2024-02-18T01:31:45.594838+0000 mon.b (mon.1) 429 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-02-18T01:31:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: cluster 2024-02-18T01:31:45.894628+0000 mgr.y (mgr.24475) 576 : cluster [DBG] pgmap v861: 228 pgs: 32 unknown, 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: audit 2024-02-18T01:31:46.594257+0000 mon.a (mon.0) 2574 : audit [INF] from='client.26878 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-92","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-02-18T01:31:46.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:46 smithi155 bash[20805]: cluster 2024-02-18T01:31:46.594472+0000 mon.a (mon.0) 2575 : cluster [DBG] osdmap e495: 8 total, 8 up, 8 in 2024-02-18T01:31:47.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[27669]: audit 2024-02-18T01:31:46.632275+0000 mon.a (mon.0) 2576 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[27669]: audit 2024-02-18T01:31:46.632300+0000 mon.b (mon.1) 430 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[27669]: audit 2024-02-18T01:31:46.633642+0000 mon.a (mon.0) 2577 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[27669]: audit 2024-02-18T01:31:46.633821+0000 mon.b (mon.1) 431 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[19734]: audit 2024-02-18T01:31:46.632275+0000 mon.a (mon.0) 2576 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[19734]: audit 2024-02-18T01:31:46.632300+0000 mon.b (mon.1) 430 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[19734]: audit 2024-02-18T01:31:46.633642+0000 mon.a (mon.0) 2577 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:47.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:47 smithi105 bash[19734]: audit 2024-02-18T01:31:46.633821+0000 mon.b (mon.1) 431 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:47 smithi155 bash[20805]: audit 2024-02-18T01:31:46.632275+0000 mon.a (mon.0) 2576 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:47 smithi155 bash[20805]: audit 2024-02-18T01:31:46.632300+0000 mon.b (mon.1) 430 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-6"}]: dispatch 2024-02-18T01:31:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:47 smithi155 bash[20805]: audit 2024-02-18T01:31:46.633642+0000 mon.a (mon.0) 2577 : audit [INF] from='client.26878 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:47 smithi155 bash[20805]: audit 2024-02-18T01:31:46.633821+0000 mon.b (mon.1) 431 : audit [INF] from='client.? v1:172.21.15.105:0/2382132472' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-6", "tierpool": "test-rados-api-smithi105-61833-92"}]: dispatch 2024-02-18T01:31:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:48.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[19734]: cluster 2024-02-18T01:31:47.618542+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T01:31:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[19734]: cluster 2024-02-18T01:31:47.895473+0000 mgr.y (mgr.24475) 577 : cluster [DBG] pgmap v864: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[19734]: audit 2024-02-18T01:31:48.042088+0000 mgr.y (mgr.24475) 578 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[27669]: cluster 2024-02-18T01:31:47.618542+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T01:31:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[27669]: cluster 2024-02-18T01:31:47.895473+0000 mgr.y (mgr.24475) 577 : cluster [DBG] pgmap v864: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:48.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:48 smithi105 bash[27669]: audit 2024-02-18T01:31:48.042088+0000 mgr.y (mgr.24475) 578 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:48 smithi155 bash[20805]: cluster 2024-02-18T01:31:47.618542+0000 mon.a (mon.0) 2578 : cluster [DBG] osdmap e496: 8 total, 8 up, 8 in 2024-02-18T01:31:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:48 smithi155 bash[20805]: cluster 2024-02-18T01:31:47.895473+0000 mgr.y (mgr.24475) 577 : cluster [DBG] pgmap v864: 196 pgs: 196 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:31:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:48 smithi155 bash[20805]: audit 2024-02-18T01:31:48.042088+0000 mgr.y (mgr.24475) 578 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[27669]: cluster 2024-02-18T01:31:48.633037+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[27669]: audit 2024-02-18T01:31:48.657492+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[27669]: audit 2024-02-18T01:31:48.658944+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[27669]: audit 2024-02-18T01:31:48.659445+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[19734]: cluster 2024-02-18T01:31:48.633037+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[19734]: audit 2024-02-18T01:31:48.657492+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[19734]: audit 2024-02-18T01:31:48.658944+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:49 smithi105 bash[19734]: audit 2024-02-18T01:31:48.659445+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:31:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:49 smithi155 bash[20805]: cluster 2024-02-18T01:31:48.633037+0000 mon.a (mon.0) 2579 : cluster [DBG] osdmap e497: 8 total, 8 up, 8 in 2024-02-18T01:31:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:49 smithi155 bash[20805]: audit 2024-02-18T01:31:48.657492+0000 mon.a (mon.0) 2580 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:49 smithi155 bash[20805]: audit 2024-02-18T01:31:48.658944+0000 mon.a (mon.0) 2581 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:49 smithi155 bash[20805]: audit 2024-02-18T01:31:48.659445+0000 mon.a (mon.0) 2582 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:31:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[19734]: audit 2024-02-18T01:31:49.642097+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:31:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[19734]: cluster 2024-02-18T01:31:49.642188+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T01:31:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[19734]: audit 2024-02-18T01:31:49.646250+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:51.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[19734]: cluster 2024-02-18T01:31:49.896238+0000 mgr.y (mgr.24475) 579 : cluster [DBG] pgmap v867: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[27669]: audit 2024-02-18T01:31:49.642097+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:31:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[27669]: cluster 2024-02-18T01:31:49.642188+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T01:31:51.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[27669]: audit 2024-02-18T01:31:49.646250+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:51.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:51 smithi105 bash[27669]: cluster 2024-02-18T01:31:49.896238+0000 mgr.y (mgr.24475) 579 : cluster [DBG] pgmap v867: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:51 smithi155 bash[20805]: audit 2024-02-18T01:31:49.642097+0000 mon.a (mon.0) 2583 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:31:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:51 smithi155 bash[20805]: cluster 2024-02-18T01:31:49.642188+0000 mon.a (mon.0) 2584 : cluster [DBG] osdmap e498: 8 total, 8 up, 8 in 2024-02-18T01:31:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:51 smithi155 bash[20805]: audit 2024-02-18T01:31:49.646250+0000 mon.a (mon.0) 2585 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:31:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:51 smithi155 bash[20805]: cluster 2024-02-18T01:31:49.896238+0000 mgr.y (mgr.24475) 579 : cluster [DBG] pgmap v867: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:52.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:31:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:31:52.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:52 smithi155 bash[20805]: cluster 2024-02-18T01:31:51.083038+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T01:31:52.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:52 smithi155 bash[20805]: audit 2024-02-18T01:31:51.516694+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:52.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:52 smithi155 bash[20805]: audit 2024-02-18T01:31:51.651904+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:31:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:52 smithi155 bash[20805]: cluster 2024-02-18T01:31:51.652082+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T01:31:52.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:52 smithi155 bash[20805]: cluster 2024-02-18T01:31:51.897034+0000 mgr.y (mgr.24475) 580 : cluster [DBG] pgmap v870: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[27669]: cluster 2024-02-18T01:31:51.083038+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T01:31:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[27669]: audit 2024-02-18T01:31:51.516694+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[27669]: audit 2024-02-18T01:31:51.651904+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[27669]: cluster 2024-02-18T01:31:51.652082+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[27669]: cluster 2024-02-18T01:31:51.897034+0000 mgr.y (mgr.24475) 580 : cluster [DBG] pgmap v870: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[19734]: cluster 2024-02-18T01:31:51.083038+0000 mon.a (mon.0) 2586 : cluster [DBG] osdmap e499: 8 total, 8 up, 8 in 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[19734]: audit 2024-02-18T01:31:51.516694+0000 mon.a (mon.0) 2587 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[19734]: audit 2024-02-18T01:31:51.651904+0000 mon.a (mon.0) 2588 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi105-61833-94", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[19734]: cluster 2024-02-18T01:31:51.652082+0000 mon.a (mon.0) 2589 : cluster [DBG] osdmap e500: 8 total, 8 up, 8 in 2024-02-18T01:31:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:52 smithi105 bash[19734]: cluster 2024-02-18T01:31:51.897034+0000 mgr.y (mgr.24475) 580 : cluster [DBG] pgmap v870: 172 pgs: 8 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:31:53.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:53 smithi105 bash[19734]: cluster 2024-02-18T01:31:52.659541+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T01:31:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:53 smithi105 bash[27669]: cluster 2024-02-18T01:31:52.659541+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T01:31:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:53 smithi155 bash[20805]: cluster 2024-02-18T01:31:52.659541+0000 mon.a (mon.0) 2590 : cluster [DBG] osdmap e501: 8 total, 8 up, 8 in 2024-02-18T01:31:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[19734]: cluster 2024-02-18T01:31:53.667224+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T01:31:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[19734]: audit 2024-02-18T01:31:53.670363+0000 mon.b (mon.1) 432 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[19734]: audit 2024-02-18T01:31:53.677146+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[19734]: cluster 2024-02-18T01:31:53.897873+0000 mgr.y (mgr.24475) 581 : cluster [DBG] pgmap v873: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[27669]: cluster 2024-02-18T01:31:53.667224+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T01:31:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[27669]: audit 2024-02-18T01:31:53.670363+0000 mon.b (mon.1) 432 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[27669]: audit 2024-02-18T01:31:53.677146+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:54 smithi105 bash[27669]: cluster 2024-02-18T01:31:53.897873+0000 mgr.y (mgr.24475) 581 : cluster [DBG] pgmap v873: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:54 smithi155 bash[20805]: cluster 2024-02-18T01:31:53.667224+0000 mon.a (mon.0) 2591 : cluster [DBG] osdmap e502: 8 total, 8 up, 8 in 2024-02-18T01:31:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:54 smithi155 bash[20805]: audit 2024-02-18T01:31:53.670363+0000 mon.b (mon.1) 432 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:54 smithi155 bash[20805]: audit 2024-02-18T01:31:53.677146+0000 mon.a (mon.0) 2592 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:54 smithi155 bash[20805]: cluster 2024-02-18T01:31:53.897873+0000 mgr.y (mgr.24475) 581 : cluster [DBG] pgmap v873: 204 pgs: 32 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:55 smithi105 bash[19734]: audit 2024-02-18T01:31:54.672293+0000 mon.a (mon.0) 2593 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:55.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:55 smithi105 bash[19734]: cluster 2024-02-18T01:31:54.672482+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T01:31:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:55 smithi105 bash[27669]: audit 2024-02-18T01:31:54.672293+0000 mon.a (mon.0) 2593 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:55.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:55 smithi105 bash[27669]: cluster 2024-02-18T01:31:54.672482+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T01:31:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:55 smithi155 bash[20805]: audit 2024-02-18T01:31:54.672293+0000 mon.a (mon.0) 2593 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:55.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:55 smithi155 bash[20805]: cluster 2024-02-18T01:31:54.672482+0000 mon.a (mon.0) 2594 : cluster [DBG] osdmap e503: 8 total, 8 up, 8 in 2024-02-18T01:31:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:31:56] "GET /metrics HTTP/1.1" 200 184942 "" "Prometheus/2.33.4" 2024-02-18T01:31:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[19734]: cluster 2024-02-18T01:31:55.700110+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T01:31:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[19734]: audit 2024-02-18T01:31:55.706320+0000 mon.b (mon.1) 433 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[19734]: audit 2024-02-18T01:31:55.706776+0000 mon.a (mon.0) 2596 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[19734]: cluster 2024-02-18T01:31:55.898772+0000 mgr.y (mgr.24475) 582 : cluster [DBG] pgmap v876: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[27669]: cluster 2024-02-18T01:31:55.700110+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T01:31:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[27669]: audit 2024-02-18T01:31:55.706320+0000 mon.b (mon.1) 433 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[27669]: audit 2024-02-18T01:31:55.706776+0000 mon.a (mon.0) 2596 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:56 smithi105 bash[27669]: cluster 2024-02-18T01:31:55.898772+0000 mgr.y (mgr.24475) 582 : cluster [DBG] pgmap v876: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:56 smithi155 bash[20805]: cluster 2024-02-18T01:31:55.700110+0000 mon.a (mon.0) 2595 : cluster [DBG] osdmap e504: 8 total, 8 up, 8 in 2024-02-18T01:31:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:56 smithi155 bash[20805]: audit 2024-02-18T01:31:55.706320+0000 mon.b (mon.1) 433 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:56 smithi155 bash[20805]: audit 2024-02-18T01:31:55.706776+0000 mon.a (mon.0) 2596 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:31:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:56 smithi155 bash[20805]: cluster 2024-02-18T01:31:55.898772+0000 mgr.y (mgr.24475) 582 : cluster [DBG] pgmap v876: 236 pgs: 64 unknown, 8 creating+peering, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:31:57.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:56.703268+0000 mon.a (mon.0) 2597 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:57.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: cluster 2024-02-18T01:31:56.703779+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:56.708181+0000 mon.b (mon.1) 434 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:56.708343+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:57.705009+0000 mon.a (mon.0) 2600 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: cluster 2024-02-18T01:31:57.705113+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:57.706376+0000 mon.a (mon.0) 2602 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[27669]: audit 2024-02-18T01:31:57.706499+0000 mon.b (mon.1) 435 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:56.703268+0000 mon.a (mon.0) 2597 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: cluster 2024-02-18T01:31:56.703779+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:56.708181+0000 mon.b (mon.1) 434 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:56.708343+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:57.705009+0000 mon.a (mon.0) 2600 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: cluster 2024-02-18T01:31:57.705113+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T01:31:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:57.706376+0000 mon.a (mon.0) 2602 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:57 smithi105 bash[19734]: audit 2024-02-18T01:31:57.706499+0000 mon.b (mon.1) 435 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:56.703268+0000 mon.a (mon.0) 2597 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-97-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: cluster 2024-02-18T01:31:56.703779+0000 mon.a (mon.0) 2598 : cluster [DBG] osdmap e505: 8 total, 8 up, 8 in 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:56.708181+0000 mon.b (mon.1) 434 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:56.708343+0000 mon.a (mon.0) 2599 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:57.705009+0000 mon.a (mon.0) 2600 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: cluster 2024-02-18T01:31:57.705113+0000 mon.a (mon.0) 2601 : cluster [DBG] osdmap e506: 8 total, 8 up, 8 in 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:57.706376+0000 mon.a (mon.0) 2602 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:57 smithi155 bash[20805]: audit 2024-02-18T01:31:57.706499+0000 mon.b (mon.1) 435 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:31:58.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: cluster 2024-02-18T01:31:57.899674+0000 mgr.y (mgr.24475) 583 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: audit 2024-02-18T01:31:58.053552+0000 mgr.y (mgr.24475) 584 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: audit 2024-02-18T01:31:58.707174+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: cluster 2024-02-18T01:31:58.707409+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: audit 2024-02-18T01:31:58.708369+0000 mon.a (mon.0) 2605 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[27669]: audit 2024-02-18T01:31:58.708612+0000 mon.b (mon.1) 436 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: cluster 2024-02-18T01:31:57.899674+0000 mgr.y (mgr.24475) 583 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: audit 2024-02-18T01:31:58.053552+0000 mgr.y (mgr.24475) 584 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:58.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: audit 2024-02-18T01:31:58.707174+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:58.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: cluster 2024-02-18T01:31:58.707409+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T01:31:58.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: audit 2024-02-18T01:31:58.708369+0000 mon.a (mon.0) 2605 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:58.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:58 smithi105 bash[19734]: audit 2024-02-18T01:31:58.708612+0000 mon.b (mon.1) 436 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: cluster 2024-02-18T01:31:57.899674+0000 mgr.y (mgr.24475) 583 : cluster [DBG] pgmap v879: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: audit 2024-02-18T01:31:58.053552+0000 mgr.y (mgr.24475) 584 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: audit 2024-02-18T01:31:58.707174+0000 mon.a (mon.0) 2603 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-97", "overlaypool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: cluster 2024-02-18T01:31:58.707409+0000 mon.a (mon.0) 2604 : cluster [DBG] osdmap e507: 8 total, 8 up, 8 in 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: audit 2024-02-18T01:31:58.708369+0000 mon.a (mon.0) 2605 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:58 smithi155 bash[20805]: audit 2024-02-18T01:31:58.708612+0000 mon.b (mon.1) 436 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:31:59.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[27669]: cluster 2024-02-18T01:31:58.712507+0000 mon.a (mon.0) 2606 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[27669]: cluster 2024-02-18T01:31:59.699837+0000 mon.a (mon.0) 2607 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[27669]: audit 2024-02-18T01:31:59.711499+0000 mon.a (mon.0) 2608 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]': finished 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[27669]: cluster 2024-02-18T01:31:59.711658+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[19734]: cluster 2024-02-18T01:31:58.712507+0000 mon.a (mon.0) 2606 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[19734]: cluster 2024-02-18T01:31:59.699837+0000 mon.a (mon.0) 2607 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[19734]: audit 2024-02-18T01:31:59.711499+0000 mon.a (mon.0) 2608 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]': finished 2024-02-18T01:31:59.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:31:59 smithi105 bash[19734]: cluster 2024-02-18T01:31:59.711658+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T01:31:59.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:59 smithi155 bash[20805]: cluster 2024-02-18T01:31:58.712507+0000 mon.a (mon.0) 2606 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:31:59.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:59 smithi155 bash[20805]: cluster 2024-02-18T01:31:59.699837+0000 mon.a (mon.0) 2607 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:31:59.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:59 smithi155 bash[20805]: audit 2024-02-18T01:31:59.711499+0000 mon.a (mon.0) 2608 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-97-cache", "mode": "writeback"}]': finished 2024-02-18T01:31:59.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:31:59 smithi155 bash[20805]: cluster 2024-02-18T01:31:59.711658+0000 mon.a (mon.0) 2609 : cluster [DBG] osdmap e508: 8 total, 8 up, 8 in 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[27669]: audit 2024-02-18T01:31:59.730413+0000 mon.a (mon.0) 2610 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[27669]: audit 2024-02-18T01:31:59.730595+0000 mon.b (mon.1) 437 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[27669]: cluster 2024-02-18T01:31:59.900535+0000 mgr.y (mgr.24475) 585 : cluster [DBG] pgmap v882: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[19734]: audit 2024-02-18T01:31:59.730413+0000 mon.a (mon.0) 2610 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[19734]: audit 2024-02-18T01:31:59.730595+0000 mon.b (mon.1) 437 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.993 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:00 smithi105 bash[19734]: cluster 2024-02-18T01:31:59.900535+0000 mgr.y (mgr.24475) 585 : cluster [DBG] pgmap v882: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:32:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:00 smithi155 bash[20805]: audit 2024-02-18T01:31:59.730413+0000 mon.a (mon.0) 2610 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:00 smithi155 bash[20805]: audit 2024-02-18T01:31:59.730595+0000 mon.b (mon.1) 437 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]: dispatch 2024-02-18T01:32:00.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:00 smithi155 bash[20805]: cluster 2024-02-18T01:31:59.900535+0000 mgr.y (mgr.24475) 585 : cluster [DBG] pgmap v882: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:32:02.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[27669]: audit 2024-02-18T01:32:00.737386+0000 mon.a (mon.0) 2611 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]': finished 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[27669]: cluster 2024-02-18T01:32:00.737473+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[27669]: audit 2024-02-18T01:32:00.740676+0000 mon.b (mon.1) 438 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[27669]: audit 2024-02-18T01:32:00.741280+0000 mon.a (mon.0) 2613 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[19734]: audit 2024-02-18T01:32:00.737386+0000 mon.a (mon.0) 2611 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]': finished 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[19734]: cluster 2024-02-18T01:32:00.737473+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[19734]: audit 2024-02-18T01:32:00.740676+0000 mon.b (mon.1) 438 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:02.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:01 smithi105 bash[19734]: audit 2024-02-18T01:32:00.741280+0000 mon.a (mon.0) 2613 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:01 smithi155 bash[20805]: audit 2024-02-18T01:32:00.737386+0000 mon.a (mon.0) 2611 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-97"}]': finished 2024-02-18T01:32:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:01 smithi155 bash[20805]: cluster 2024-02-18T01:32:00.737473+0000 mon.a (mon.0) 2612 : cluster [DBG] osdmap e509: 8 total, 8 up, 8 in 2024-02-18T01:32:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:01 smithi155 bash[20805]: audit 2024-02-18T01:32:00.740676+0000 mon.b (mon.1) 438 : audit [INF] from='client.? v1:172.21.15.105:0/3936894258' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:02.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:01 smithi155 bash[20805]: audit 2024-02-18T01:32:00.741280+0000 mon.a (mon.0) 2613 : audit [INF] from='client.48976 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]: dispatch 2024-02-18T01:32:03.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[19734]: cluster 2024-02-18T01:32:01.732396+0000 mon.a (mon.0) 2614 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[19734]: audit 2024-02-18T01:32:01.741752+0000 mon.a (mon.0) 2615 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[19734]: cluster 2024-02-18T01:32:01.741963+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[19734]: cluster 2024-02-18T01:32:01.901451+0000 mgr.y (mgr.24475) 586 : cluster [DBG] pgmap v885: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[27669]: cluster 2024-02-18T01:32:01.732396+0000 mon.a (mon.0) 2614 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[27669]: audit 2024-02-18T01:32:01.741752+0000 mon.a (mon.0) 2615 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[27669]: cluster 2024-02-18T01:32:01.741963+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T01:32:03.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:02 smithi105 bash[27669]: cluster 2024-02-18T01:32:01.901451+0000 mgr.y (mgr.24475) 586 : cluster [DBG] pgmap v885: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:02 smithi155 bash[20805]: cluster 2024-02-18T01:32:01.732396+0000 mon.a (mon.0) 2614 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:02 smithi155 bash[20805]: audit 2024-02-18T01:32:01.741752+0000 mon.a (mon.0) 2615 : audit [INF] from='client.48976 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-97", "tierpool": "test-rados-api-smithi105-61833-97-cache"}]': finished 2024-02-18T01:32:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:02 smithi155 bash[20805]: cluster 2024-02-18T01:32:01.741963+0000 mon.a (mon.0) 2616 : cluster [DBG] osdmap e510: 8 total, 8 up, 8 in 2024-02-18T01:32:03.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:02 smithi155 bash[20805]: cluster 2024-02-18T01:32:01.901451+0000 mgr.y (mgr.24475) 586 : cluster [DBG] pgmap v885: 236 pgs: 236 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:04.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:03 smithi105 bash[27669]: cluster 2024-02-18T01:32:02.746612+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T01:32:04.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:03 smithi105 bash[27669]: cluster 2024-02-18T01:32:03.751114+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T01:32:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:03 smithi105 bash[19734]: cluster 2024-02-18T01:32:02.746612+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T01:32:04.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:03 smithi105 bash[19734]: cluster 2024-02-18T01:32:03.751114+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T01:32:04.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:03 smithi155 bash[20805]: cluster 2024-02-18T01:32:02.746612+0000 mon.a (mon.0) 2617 : cluster [DBG] osdmap e511: 8 total, 8 up, 8 in 2024-02-18T01:32:04.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:03 smithi155 bash[20805]: cluster 2024-02-18T01:32:03.751114+0000 mon.a (mon.0) 2618 : cluster [DBG] osdmap e512: 8 total, 8 up, 8 in 2024-02-18T01:32:05.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.772021+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.772369+0000 mon.a (mon.0) 2619 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.773756+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.774042+0000 mon.a (mon.0) 2620 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.774547+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: audit 2024-02-18T01:32:03.774815+0000 mon.a (mon.0) 2621 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[27669]: cluster 2024-02-18T01:32:03.902276+0000 mgr.y (mgr.24475) 587 : cluster [DBG] pgmap v888: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.772021+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.772369+0000 mon.a (mon.0) 2619 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.773756+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.774042+0000 mon.a (mon.0) 2620 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.774547+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: audit 2024-02-18T01:32:03.774815+0000 mon.a (mon.0) 2621 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:04 smithi105 bash[19734]: cluster 2024-02-18T01:32:03.902276+0000 mgr.y (mgr.24475) 587 : cluster [DBG] pgmap v888: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.772021+0000 mon.c (mon.2) 399 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.772369+0000 mon.a (mon.0) 2619 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.773756+0000 mon.c (mon.2) 400 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.774042+0000 mon.a (mon.0) 2620 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.774547+0000 mon.c (mon.2) 401 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: audit 2024-02-18T01:32:03.774815+0000 mon.a (mon.0) 2621 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:05.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:04 smithi155 bash[20805]: cluster 2024-02-18T01:32:03.902276+0000 mgr.y (mgr.24475) 587 : cluster [DBG] pgmap v888: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[19734]: audit 2024-02-18T01:32:04.777003+0000 mon.a (mon.0) 2622 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[19734]: cluster 2024-02-18T01:32:04.777157+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[19734]: audit 2024-02-18T01:32:04.778070+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[19734]: audit 2024-02-18T01:32:04.780145+0000 mon.a (mon.0) 2624 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[27669]: audit 2024-02-18T01:32:04.777003+0000 mon.a (mon.0) 2622 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[27669]: cluster 2024-02-18T01:32:04.777157+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T01:32:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[27669]: audit 2024-02-18T01:32:04.778070+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[27669]: audit 2024-02-18T01:32:04.780145+0000 mon.a (mon.0) 2624 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:06 smithi155 bash[20805]: audit 2024-02-18T01:32:04.777003+0000 mon.a (mon.0) 2622 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-99", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:06 smithi155 bash[20805]: cluster 2024-02-18T01:32:04.777157+0000 mon.a (mon.0) 2623 : cluster [DBG] osdmap e513: 8 total, 8 up, 8 in 2024-02-18T01:32:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:06 smithi155 bash[20805]: audit 2024-02-18T01:32:04.778070+0000 mon.c (mon.2) 402 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:06 smithi155 bash[20805]: audit 2024-02-18T01:32:04.780145+0000 mon.a (mon.0) 2624 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:06] "GET /metrics HTTP/1.1" 200 187475 "" "Prometheus/2.33.4" 2024-02-18T01:32:07.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[27669]: cluster 2024-02-18T01:32:05.902955+0000 mgr.y (mgr.24475) 588 : cluster [DBG] pgmap v891: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[27669]: cluster 2024-02-18T01:32:06.213100+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[27669]: audit 2024-02-18T01:32:06.790552+0000 mon.a (mon.0) 2626 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[27669]: cluster 2024-02-18T01:32:06.790906+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[19734]: cluster 2024-02-18T01:32:05.902955+0000 mgr.y (mgr.24475) 588 : cluster [DBG] pgmap v891: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[19734]: cluster 2024-02-18T01:32:06.213100+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[19734]: audit 2024-02-18T01:32:06.790552+0000 mon.a (mon.0) 2626 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:07 smithi105 bash[19734]: cluster 2024-02-18T01:32:06.790906+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T01:32:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:07 smithi155 bash[20805]: cluster 2024-02-18T01:32:05.902955+0000 mgr.y (mgr.24475) 588 : cluster [DBG] pgmap v891: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:07 smithi155 bash[20805]: cluster 2024-02-18T01:32:06.213100+0000 mon.a (mon.0) 2625 : cluster [DBG] osdmap e514: 8 total, 8 up, 8 in 2024-02-18T01:32:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:07 smithi155 bash[20805]: audit 2024-02-18T01:32:06.790552+0000 mon.a (mon.0) 2626 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-99", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:07 smithi155 bash[20805]: cluster 2024-02-18T01:32:06.790906+0000 mon.a (mon.0) 2627 : cluster [DBG] osdmap e515: 8 total, 8 up, 8 in 2024-02-18T01:32:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:09.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[27669]: cluster 2024-02-18T01:32:07.794977+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[27669]: audit 2024-02-18T01:32:07.796426+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[27669]: audit 2024-02-18T01:32:07.797555+0000 mon.a (mon.0) 2629 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[27669]: cluster 2024-02-18T01:32:07.903794+0000 mgr.y (mgr.24475) 589 : cluster [DBG] pgmap v894: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[27669]: audit 2024-02-18T01:32:08.061235+0000 mgr.y (mgr.24475) 590 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[19734]: cluster 2024-02-18T01:32:07.794977+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[19734]: audit 2024-02-18T01:32:07.796426+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[19734]: audit 2024-02-18T01:32:07.797555+0000 mon.a (mon.0) 2629 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[19734]: cluster 2024-02-18T01:32:07.903794+0000 mgr.y (mgr.24475) 589 : cluster [DBG] pgmap v894: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:09.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:08 smithi105 bash[19734]: audit 2024-02-18T01:32:08.061235+0000 mgr.y (mgr.24475) 590 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[20805]: cluster 2024-02-18T01:32:07.794977+0000 mon.a (mon.0) 2628 : cluster [DBG] osdmap e516: 8 total, 8 up, 8 in 2024-02-18T01:32:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[20805]: audit 2024-02-18T01:32:07.796426+0000 mon.c (mon.2) 403 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[20805]: audit 2024-02-18T01:32:07.797555+0000 mon.a (mon.0) 2629 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[20805]: cluster 2024-02-18T01:32:07.903794+0000 mgr.y (mgr.24475) 589 : cluster [DBG] pgmap v894: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:09.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:08 smithi155 bash[20805]: audit 2024-02-18T01:32:08.061235+0000 mgr.y (mgr.24475) 590 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:10.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:08.798096+0000 mon.a (mon.0) 2630 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: cluster 2024-02-18T01:32:08.798417+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:08.799809+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:08.802408+0000 mon.a (mon.0) 2632 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:09.799395+0000 mon.a (mon.0) 2633 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: cluster 2024-02-18T01:32:09.799581+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:09.800299+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[19734]: audit 2024-02-18T01:32:09.801091+0000 mon.a (mon.0) 2635 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:08.798096+0000 mon.a (mon.0) 2630 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:10.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: cluster 2024-02-18T01:32:08.798417+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:08.799809+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:08.802408+0000 mon.a (mon.0) 2632 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:09.799395+0000 mon.a (mon.0) 2633 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: cluster 2024-02-18T01:32:09.799581+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:09.800299+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:09 smithi105 bash[27669]: audit 2024-02-18T01:32:09.801091+0000 mon.a (mon.0) 2635 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:08.798096+0000 mon.a (mon.0) 2630 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-99-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: cluster 2024-02-18T01:32:08.798417+0000 mon.a (mon.0) 2631 : cluster [DBG] osdmap e517: 8 total, 8 up, 8 in 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:08.799809+0000 mon.c (mon.2) 404 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:08.802408+0000 mon.a (mon.0) 2632 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:09.799395+0000 mon.a (mon.0) 2633 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: cluster 2024-02-18T01:32:09.799581+0000 mon.a (mon.0) 2634 : cluster [DBG] osdmap e518: 8 total, 8 up, 8 in 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:09.800299+0000 mon.c (mon.2) 405 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:10.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:09 smithi155 bash[20805]: audit 2024-02-18T01:32:09.801091+0000 mon.a (mon.0) 2635 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[19734]: cluster 2024-02-18T01:32:09.904632+0000 mgr.y (mgr.24475) 591 : cluster [DBG] pgmap v897: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[19734]: audit 2024-02-18T01:32:10.801291+0000 mon.a (mon.0) 2636 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[19734]: cluster 2024-02-18T01:32:10.801508+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[19734]: audit 2024-02-18T01:32:10.802408+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[19734]: audit 2024-02-18T01:32:10.802861+0000 mon.a (mon.0) 2638 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[27669]: cluster 2024-02-18T01:32:09.904632+0000 mgr.y (mgr.24475) 591 : cluster [DBG] pgmap v897: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[27669]: audit 2024-02-18T01:32:10.801291+0000 mon.a (mon.0) 2636 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[27669]: cluster 2024-02-18T01:32:10.801508+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T01:32:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[27669]: audit 2024-02-18T01:32:10.802408+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:11.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:10 smithi105 bash[27669]: audit 2024-02-18T01:32:10.802861+0000 mon.a (mon.0) 2638 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:10 smithi155 bash[20805]: cluster 2024-02-18T01:32:09.904632+0000 mgr.y (mgr.24475) 591 : cluster [DBG] pgmap v897: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:10 smithi155 bash[20805]: audit 2024-02-18T01:32:10.801291+0000 mon.a (mon.0) 2636 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-99", "overlaypool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:10 smithi155 bash[20805]: cluster 2024-02-18T01:32:10.801508+0000 mon.a (mon.0) 2637 : cluster [DBG] osdmap e519: 8 total, 8 up, 8 in 2024-02-18T01:32:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:10 smithi155 bash[20805]: audit 2024-02-18T01:32:10.802408+0000 mon.c (mon.2) 406 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:10 smithi155 bash[20805]: audit 2024-02-18T01:32:10.802861+0000 mon.a (mon.0) 2638 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]: dispatch 2024-02-18T01:32:12.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[19734]: cluster 2024-02-18T01:32:11.795292+0000 mon.a (mon.0) 2639 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[19734]: audit 2024-02-18T01:32:11.805728+0000 mon.a (mon.0) 2640 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]': finished 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[19734]: cluster 2024-02-18T01:32:11.805948+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[19734]: audit 2024-02-18T01:32:11.809448+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[19734]: audit 2024-02-18T01:32:11.809962+0000 mon.a (mon.0) 2642 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[27669]: cluster 2024-02-18T01:32:11.795292+0000 mon.a (mon.0) 2639 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[27669]: audit 2024-02-18T01:32:11.805728+0000 mon.a (mon.0) 2640 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]': finished 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[27669]: cluster 2024-02-18T01:32:11.805948+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[27669]: audit 2024-02-18T01:32:11.809448+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:11 smithi105 bash[27669]: audit 2024-02-18T01:32:11.809962+0000 mon.a (mon.0) 2642 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[20805]: cluster 2024-02-18T01:32:11.795292+0000 mon.a (mon.0) 2639 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[20805]: audit 2024-02-18T01:32:11.805728+0000 mon.a (mon.0) 2640 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-99-cache", "mode": "writeback"}]': finished 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[20805]: cluster 2024-02-18T01:32:11.805948+0000 mon.a (mon.0) 2641 : cluster [DBG] osdmap e520: 8 total, 8 up, 8 in 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[20805]: audit 2024-02-18T01:32:11.809448+0000 mon.c (mon.2) 407 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:11 smithi155 bash[20805]: audit 2024-02-18T01:32:11.809962+0000 mon.a (mon.0) 2642 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:32:13.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[19734]: cluster 2024-02-18T01:32:11.905538+0000 mgr.y (mgr.24475) 592 : cluster [DBG] pgmap v900: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[19734]: audit 2024-02-18T01:32:12.807631+0000 mon.a (mon.0) 2643 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[19734]: cluster 2024-02-18T01:32:12.807886+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[19734]: audit 2024-02-18T01:32:12.809048+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[19734]: audit 2024-02-18T01:32:12.809840+0000 mon.a (mon.0) 2645 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[27669]: cluster 2024-02-18T01:32:11.905538+0000 mgr.y (mgr.24475) 592 : cluster [DBG] pgmap v900: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[27669]: audit 2024-02-18T01:32:12.807631+0000 mon.a (mon.0) 2643 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[27669]: cluster 2024-02-18T01:32:12.807886+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[27669]: audit 2024-02-18T01:32:12.809048+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:12 smithi105 bash[27669]: audit 2024-02-18T01:32:12.809840+0000 mon.a (mon.0) 2645 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:12 smithi155 bash[20805]: cluster 2024-02-18T01:32:11.905538+0000 mgr.y (mgr.24475) 592 : cluster [DBG] pgmap v900: 212 pgs: 32 unknown, 8 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:12 smithi155 bash[20805]: audit 2024-02-18T01:32:12.807631+0000 mon.a (mon.0) 2643 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:32:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:12 smithi155 bash[20805]: cluster 2024-02-18T01:32:12.807886+0000 mon.a (mon.0) 2644 : cluster [DBG] osdmap e521: 8 total, 8 up, 8 in 2024-02-18T01:32:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:12 smithi155 bash[20805]: audit 2024-02-18T01:32:12.809048+0000 mon.c (mon.2) 408 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:12 smithi155 bash[20805]: audit 2024-02-18T01:32:12.809840+0000 mon.a (mon.0) 2645 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:32:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[19734]: audit 2024-02-18T01:32:13.808517+0000 mon.a (mon.0) 2646 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[19734]: cluster 2024-02-18T01:32:13.809190+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[19734]: audit 2024-02-18T01:32:13.812517+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[19734]: audit 2024-02-18T01:32:13.812935+0000 mon.a (mon.0) 2648 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[19734]: cluster 2024-02-18T01:32:13.906325+0000 mgr.y (mgr.24475) 593 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[27669]: audit 2024-02-18T01:32:13.808517+0000 mon.a (mon.0) 2646 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[27669]: cluster 2024-02-18T01:32:13.809190+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[27669]: audit 2024-02-18T01:32:13.812517+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[27669]: audit 2024-02-18T01:32:13.812935+0000 mon.a (mon.0) 2648 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:14 smithi105 bash[27669]: cluster 2024-02-18T01:32:13.906325+0000 mgr.y (mgr.24475) 593 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:14 smithi155 bash[20805]: audit 2024-02-18T01:32:13.808517+0000 mon.a (mon.0) 2646 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:32:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:14 smithi155 bash[20805]: cluster 2024-02-18T01:32:13.809190+0000 mon.a (mon.0) 2647 : cluster [DBG] osdmap e522: 8 total, 8 up, 8 in 2024-02-18T01:32:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:14 smithi155 bash[20805]: audit 2024-02-18T01:32:13.812517+0000 mon.c (mon.2) 409 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:14 smithi155 bash[20805]: audit 2024-02-18T01:32:13.812935+0000 mon.a (mon.0) 2648 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:32:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:14 smithi155 bash[20805]: cluster 2024-02-18T01:32:13.906325+0000 mgr.y (mgr.24475) 593 : cluster [DBG] pgmap v903: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:16.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[27669]: cluster 2024-02-18T01:32:14.803037+0000 mon.a (mon.0) 2649 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:16.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[27669]: audit 2024-02-18T01:32:14.812298+0000 mon.a (mon.0) 2650 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[27669]: cluster 2024-02-18T01:32:14.812507+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[27669]: audit 2024-02-18T01:32:14.816509+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[27669]: audit 2024-02-18T01:32:14.821718+0000 mon.a (mon.0) 2652 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[19734]: cluster 2024-02-18T01:32:14.803037+0000 mon.a (mon.0) 2649 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[19734]: audit 2024-02-18T01:32:14.812298+0000 mon.a (mon.0) 2650 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[19734]: cluster 2024-02-18T01:32:14.812507+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[19734]: audit 2024-02-18T01:32:14.816509+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:15 smithi105 bash[19734]: audit 2024-02-18T01:32:14.821718+0000 mon.a (mon.0) 2652 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:15 smithi155 bash[20805]: cluster 2024-02-18T01:32:14.803037+0000 mon.a (mon.0) 2649 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:15 smithi155 bash[20805]: audit 2024-02-18T01:32:14.812298+0000 mon.a (mon.0) 2650 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:32:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:15 smithi155 bash[20805]: cluster 2024-02-18T01:32:14.812507+0000 mon.a (mon.0) 2651 : cluster [DBG] osdmap e523: 8 total, 8 up, 8 in 2024-02-18T01:32:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:15 smithi155 bash[20805]: audit 2024-02-18T01:32:14.816509+0000 mon.c (mon.2) 410 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:15 smithi155 bash[20805]: audit 2024-02-18T01:32:14.821718+0000 mon.a (mon.0) 2652 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-02-18T01:32:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:16] "GET /metrics HTTP/1.1" 200 187475 "" "Prometheus/2.33.4" 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[27669]: audit 2024-02-18T01:32:15.826561+0000 mon.a (mon.0) 2653 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[27669]: cluster 2024-02-18T01:32:15.826780+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[27669]: audit 2024-02-18T01:32:15.874926+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[27669]: audit 2024-02-18T01:32:15.875356+0000 mon.a (mon.0) 2655 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[27669]: cluster 2024-02-18T01:32:15.907250+0000 mgr.y (mgr.24475) 594 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[19734]: audit 2024-02-18T01:32:15.826561+0000 mon.a (mon.0) 2653 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T01:32:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[19734]: cluster 2024-02-18T01:32:15.826780+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T01:32:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[19734]: audit 2024-02-18T01:32:15.874926+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[19734]: audit 2024-02-18T01:32:15.875356+0000 mon.a (mon.0) 2655 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:16.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:16 smithi105 bash[19734]: cluster 2024-02-18T01:32:15.907250+0000 mgr.y (mgr.24475) 594 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:16 smithi155 bash[20805]: audit 2024-02-18T01:32:15.826561+0000 mon.a (mon.0) 2653 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-99-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-02-18T01:32:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:16 smithi155 bash[20805]: cluster 2024-02-18T01:32:15.826780+0000 mon.a (mon.0) 2654 : cluster [DBG] osdmap e524: 8 total, 8 up, 8 in 2024-02-18T01:32:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:16 smithi155 bash[20805]: audit 2024-02-18T01:32:15.874926+0000 mon.c (mon.2) 411 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:16 smithi155 bash[20805]: audit 2024-02-18T01:32:15.875356+0000 mon.a (mon.0) 2655 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:16 smithi155 bash[20805]: cluster 2024-02-18T01:32:15.907250+0000 mgr.y (mgr.24475) 594 : cluster [DBG] pgmap v906: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[19734]: audit 2024-02-18T01:32:16.837946+0000 mon.a (mon.0) 2656 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[19734]: cluster 2024-02-18T01:32:16.838201+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[19734]: audit 2024-02-18T01:32:16.842211+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[19734]: audit 2024-02-18T01:32:16.847859+0000 mon.a (mon.0) 2658 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[27669]: audit 2024-02-18T01:32:16.837946+0000 mon.a (mon.0) 2656 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[27669]: cluster 2024-02-18T01:32:16.838201+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[27669]: audit 2024-02-18T01:32:16.842211+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:17 smithi105 bash[27669]: audit 2024-02-18T01:32:16.847859+0000 mon.a (mon.0) 2658 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:18.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:17 smithi155 bash[20805]: audit 2024-02-18T01:32:16.837946+0000 mon.a (mon.0) 2656 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:17 smithi155 bash[20805]: cluster 2024-02-18T01:32:16.838201+0000 mon.a (mon.0) 2657 : cluster [DBG] osdmap e525: 8 total, 8 up, 8 in 2024-02-18T01:32:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:17 smithi155 bash[20805]: audit 2024-02-18T01:32:16.842211+0000 mon.c (mon.2) 412 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:17 smithi155 bash[20805]: audit 2024-02-18T01:32:16.847859+0000 mon.a (mon.0) 2658 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]: dispatch 2024-02-18T01:32:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[19734]: audit 2024-02-18T01:32:17.862438+0000 mon.a (mon.0) 2659 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[19734]: cluster 2024-02-18T01:32:17.862545+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[19734]: cluster 2024-02-18T01:32:17.907762+0000 mgr.y (mgr.24475) 595 : cluster [DBG] pgmap v909: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[19734]: audit 2024-02-18T01:32:18.069597+0000 mgr.y (mgr.24475) 596 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[27669]: audit 2024-02-18T01:32:17.862438+0000 mon.a (mon.0) 2659 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[27669]: cluster 2024-02-18T01:32:17.862545+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[27669]: cluster 2024-02-18T01:32:17.907762+0000 mgr.y (mgr.24475) 595 : cluster [DBG] pgmap v909: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:32:19.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:18 smithi105 bash[27669]: audit 2024-02-18T01:32:18.069597+0000 mgr.y (mgr.24475) 596 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:18 smithi155 bash[20805]: audit 2024-02-18T01:32:17.862438+0000 mon.a (mon.0) 2659 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-99", "tierpool": "test-rados-api-smithi105-61833-99-cache"}]': finished 2024-02-18T01:32:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:18 smithi155 bash[20805]: cluster 2024-02-18T01:32:17.862545+0000 mon.a (mon.0) 2660 : cluster [DBG] osdmap e526: 8 total, 8 up, 8 in 2024-02-18T01:32:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:18 smithi155 bash[20805]: cluster 2024-02-18T01:32:17.907762+0000 mgr.y (mgr.24475) 595 : cluster [DBG] pgmap v909: 212 pgs: 212 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:32:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:18 smithi155 bash[20805]: audit 2024-02-18T01:32:18.069597+0000 mgr.y (mgr.24475) 596 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:19 smithi105 bash[19734]: cluster 2024-02-18T01:32:18.856778+0000 mon.a (mon.0) 2661 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:19 smithi105 bash[19734]: cluster 2024-02-18T01:32:18.863973+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T01:32:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:19 smithi105 bash[27669]: cluster 2024-02-18T01:32:18.856778+0000 mon.a (mon.0) 2661 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:19 smithi105 bash[27669]: cluster 2024-02-18T01:32:18.863973+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T01:32:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:19 smithi155 bash[20805]: cluster 2024-02-18T01:32:18.856778+0000 mon.a (mon.0) 2661 : cluster [WRN] Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:19 smithi155 bash[20805]: cluster 2024-02-18T01:32:18.863973+0000 mon.a (mon.0) 2662 : cluster [DBG] osdmap e527: 8 total, 8 up, 8 in 2024-02-18T01:32:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[19734]: cluster 2024-02-18T01:32:19.889357+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[19734]: audit 2024-02-18T01:32:19.890676+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[19734]: audit 2024-02-18T01:32:19.895759+0000 mon.a (mon.0) 2664 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[19734]: cluster 2024-02-18T01:32:19.908230+0000 mgr.y (mgr.24475) 597 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[27669]: cluster 2024-02-18T01:32:19.889357+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[27669]: audit 2024-02-18T01:32:19.890676+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[27669]: audit 2024-02-18T01:32:19.895759+0000 mon.a (mon.0) 2664 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:20 smithi105 bash[27669]: cluster 2024-02-18T01:32:19.908230+0000 mgr.y (mgr.24475) 597 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:20 smithi155 bash[20805]: cluster 2024-02-18T01:32:19.889357+0000 mon.a (mon.0) 2663 : cluster [DBG] osdmap e528: 8 total, 8 up, 8 in 2024-02-18T01:32:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:20 smithi155 bash[20805]: audit 2024-02-18T01:32:19.890676+0000 mon.c (mon.2) 413 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:20 smithi155 bash[20805]: audit 2024-02-18T01:32:19.895759+0000 mon.a (mon.0) 2664 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:20 smithi155 bash[20805]: cluster 2024-02-18T01:32:19.908230+0000 mgr.y (mgr.24475) 597 : cluster [DBG] pgmap v912: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:22.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[27669]: audit 2024-02-18T01:32:20.903602+0000 mon.a (mon.0) 2665 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:22.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[27669]: cluster 2024-02-18T01:32:20.903747+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[27669]: audit 2024-02-18T01:32:20.904480+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[27669]: audit 2024-02-18T01:32:20.905160+0000 mon.a (mon.0) 2667 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[19734]: audit 2024-02-18T01:32:20.903602+0000 mon.a (mon.0) 2665 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[19734]: cluster 2024-02-18T01:32:20.903747+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[19734]: audit 2024-02-18T01:32:20.904480+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:22.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:21 smithi105 bash[19734]: audit 2024-02-18T01:32:20.905160+0000 mon.a (mon.0) 2667 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:21 smithi155 bash[20805]: audit 2024-02-18T01:32:20.903602+0000 mon.a (mon.0) 2665 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:21 smithi155 bash[20805]: cluster 2024-02-18T01:32:20.903747+0000 mon.a (mon.0) 2666 : cluster [DBG] osdmap e529: 8 total, 8 up, 8 in 2024-02-18T01:32:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:21 smithi155 bash[20805]: audit 2024-02-18T01:32:20.904480+0000 mon.c (mon.2) 414 : audit [INF] from='client.? v1:172.21.15.105:0/700041850' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:21 smithi155 bash[20805]: audit 2024-02-18T01:32:20.905160+0000 mon.a (mon.0) 2667 : audit [INF] from='client.48257 ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]: dispatch 2024-02-18T01:32:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[19734]: audit 2024-02-18T01:32:21.903002+0000 mon.a (mon.0) 2668 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[19734]: cluster 2024-02-18T01:32:21.903204+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T01:32:23.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[19734]: cluster 2024-02-18T01:32:21.908660+0000 mgr.y (mgr.24475) 598 : cluster [DBG] pgmap v915: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[27669]: audit 2024-02-18T01:32:21.903002+0000 mon.a (mon.0) 2668 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[27669]: cluster 2024-02-18T01:32:21.903204+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T01:32:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:22 smithi105 bash[27669]: cluster 2024-02-18T01:32:21.908660+0000 mgr.y (mgr.24475) 598 : cluster [DBG] pgmap v915: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:22 smithi155 bash[20805]: audit 2024-02-18T01:32:21.903002+0000 mon.a (mon.0) 2668 : audit [INF] from='client.48257 ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-99"}]': finished 2024-02-18T01:32:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:22 smithi155 bash[20805]: cluster 2024-02-18T01:32:21.903204+0000 mon.a (mon.0) 2669 : cluster [DBG] osdmap e530: 8 total, 8 up, 8 in 2024-02-18T01:32:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:22 smithi155 bash[20805]: cluster 2024-02-18T01:32:21.908660+0000 mgr.y (mgr.24475) 598 : cluster [DBG] pgmap v915: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:24.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:23 smithi105 bash[27669]: cluster 2024-02-18T01:32:22.921366+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T01:32:24.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:23 smithi105 bash[27669]: audit 2024-02-18T01:32:22.923241+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:23 smithi105 bash[19734]: cluster 2024-02-18T01:32:22.921366+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T01:32:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:23 smithi105 bash[19734]: audit 2024-02-18T01:32:22.923241+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:23 smithi155 bash[20805]: cluster 2024-02-18T01:32:22.921366+0000 mon.a (mon.0) 2670 : cluster [DBG] osdmap e531: 8 total, 8 up, 8 in 2024-02-18T01:32:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:23 smithi155 bash[20805]: audit 2024-02-18T01:32:22.923241+0000 mon.a (mon.0) 2671 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:25.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[19734]: cluster 2024-02-18T01:32:23.909320+0000 mgr.y (mgr.24475) 599 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[19734]: cluster 2024-02-18T01:32:23.915891+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[19734]: audit 2024-02-18T01:32:23.929367+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[19734]: cluster 2024-02-18T01:32:23.929445+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[19734]: audit 2024-02-18T01:32:23.930594+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[27669]: cluster 2024-02-18T01:32:23.909320+0000 mgr.y (mgr.24475) 599 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[27669]: cluster 2024-02-18T01:32:23.915891+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:25.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[27669]: audit 2024-02-18T01:32:23.929367+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:25.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[27669]: cluster 2024-02-18T01:32:23.929445+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T01:32:25.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:24 smithi105 bash[27669]: audit 2024-02-18T01:32:23.930594+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:24 smithi155 bash[20805]: cluster 2024-02-18T01:32:23.909320+0000 mgr.y (mgr.24475) 599 : cluster [DBG] pgmap v917: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:24 smithi155 bash[20805]: cluster 2024-02-18T01:32:23.915891+0000 mon.a (mon.0) 2672 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:24 smithi155 bash[20805]: audit 2024-02-18T01:32:23.929367+0000 mon.a (mon.0) 2673 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:24 smithi155 bash[20805]: cluster 2024-02-18T01:32:23.929445+0000 mon.a (mon.0) 2674 : cluster [DBG] osdmap e532: 8 total, 8 up, 8 in 2024-02-18T01:32:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:24 smithi155 bash[20805]: audit 2024-02-18T01:32:23.930594+0000 mon.a (mon.0) 2675 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]: dispatch 2024-02-18T01:32:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[19734]: audit 2024-02-18T01:32:24.943575+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[19734]: cluster 2024-02-18T01:32:24.943723+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T01:32:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[19734]: audit 2024-02-18T01:32:24.963572+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[19734]: audit 2024-02-18T01:32:24.965335+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[19734]: audit 2024-02-18T01:32:24.965872+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[27669]: audit 2024-02-18T01:32:24.943575+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[27669]: cluster 2024-02-18T01:32:24.943723+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T01:32:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[27669]: audit 2024-02-18T01:32:24.963572+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[27669]: audit 2024-02-18T01:32:24.965335+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:25 smithi105 bash[27669]: audit 2024-02-18T01:32:24.965872+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:25 smithi155 bash[20805]: audit 2024-02-18T01:32:24.943575+0000 mon.a (mon.0) 2676 : audit [INF] from='client.? v1:172.21.15.105:0/296277384' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi105-61833-94"}]': finished 2024-02-18T01:32:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:25 smithi155 bash[20805]: cluster 2024-02-18T01:32:24.943723+0000 mon.a (mon.0) 2677 : cluster [DBG] osdmap e533: 8 total, 8 up, 8 in 2024-02-18T01:32:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:25 smithi155 bash[20805]: audit 2024-02-18T01:32:24.963572+0000 mon.a (mon.0) 2678 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:25 smithi155 bash[20805]: audit 2024-02-18T01:32:24.965335+0000 mon.a (mon.0) 2679 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:25 smithi155 bash[20805]: audit 2024-02-18T01:32:24.965872+0000 mon.a (mon.0) 2680 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-02-18T01:32:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:26] "GET /metrics HTTP/1.1" 200 187470 "" "Prometheus/2.33.4" 2024-02-18T01:32:27.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: cluster 2024-02-18T01:32:25.910085+0000 mgr.y (mgr.24475) 600 : cluster [DBG] pgmap v920: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:27.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:25.958904+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:27.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: cluster 2024-02-18T01:32:25.959004+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T01:32:27.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:25.959693+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:27.742 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:26.444078+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:26.764198+0000 mon.a (mon.0) 2685 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:26.765411+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[19734]: audit 2024-02-18T01:32:26.773758+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: cluster 2024-02-18T01:32:25.910085+0000 mgr.y (mgr.24475) 600 : cluster [DBG] pgmap v920: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:25.958904+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:27.743 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: cluster 2024-02-18T01:32:25.959004+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T01:32:27.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:25.959693+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:27.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:26.444078+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:32:27.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:26.764198+0000 mon.a (mon.0) 2685 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:32:27.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:26.765411+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:32:27.744 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:27 smithi105 bash[27669]: audit 2024-02-18T01:32:26.773758+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: cluster 2024-02-18T01:32:25.910085+0000 mgr.y (mgr.24475) 600 : cluster [DBG] pgmap v920: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:25.958904+0000 mon.a (mon.0) 2681 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi105-61833-101", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: cluster 2024-02-18T01:32:25.959004+0000 mon.a (mon.0) 2682 : cluster [DBG] osdmap e534: 8 total, 8 up, 8 in 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:25.959693+0000 mon.a (mon.0) 2683 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:26.444078+0000 mon.a (mon.0) 2684 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:26.764198+0000 mon.a (mon.0) 2685 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:32:27.745 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:26.765411+0000 mon.a (mon.0) 2686 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:32:27.746 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:27 smithi155 bash[20805]: audit 2024-02-18T01:32:26.773758+0000 mon.a (mon.0) 2687 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:28.404 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[19734]: cluster 2024-02-18T01:32:27.390177+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T01:32:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[19734]: audit 2024-02-18T01:32:27.965556+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:28.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[19734]: cluster 2024-02-18T01:32:27.965766+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T01:32:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[27669]: cluster 2024-02-18T01:32:27.390177+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T01:32:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[27669]: audit 2024-02-18T01:32:27.965556+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:28 smithi105 bash[27669]: cluster 2024-02-18T01:32:27.965766+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T01:32:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:28 smithi155 bash[20805]: cluster 2024-02-18T01:32:27.390177+0000 mon.a (mon.0) 2688 : cluster [DBG] osdmap e535: 8 total, 8 up, 8 in 2024-02-18T01:32:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:28 smithi155 bash[20805]: audit 2024-02-18T01:32:27.965556+0000 mon.a (mon.0) 2689 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi105-61833-101", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:28 smithi155 bash[20805]: cluster 2024-02-18T01:32:27.965766+0000 mon.a (mon.0) 2690 : cluster [DBG] osdmap e536: 8 total, 8 up, 8 in 2024-02-18T01:32:29.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[19734]: cluster 2024-02-18T01:32:27.911872+0000 mgr.y (mgr.24475) 601 : cluster [DBG] pgmap v923: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:29.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[19734]: audit 2024-02-18T01:32:28.073622+0000 mgr.y (mgr.24475) 602 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[19734]: cluster 2024-02-18T01:32:28.973077+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[19734]: audit 2024-02-18T01:32:28.977264+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[27669]: cluster 2024-02-18T01:32:27.911872+0000 mgr.y (mgr.24475) 601 : cluster [DBG] pgmap v923: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[27669]: audit 2024-02-18T01:32:28.073622+0000 mgr.y (mgr.24475) 602 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[27669]: cluster 2024-02-18T01:32:28.973077+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T01:32:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:29 smithi105 bash[27669]: audit 2024-02-18T01:32:28.977264+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:29 smithi155 bash[20805]: cluster 2024-02-18T01:32:27.911872+0000 mgr.y (mgr.24475) 601 : cluster [DBG] pgmap v923: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:29 smithi155 bash[20805]: audit 2024-02-18T01:32:28.073622+0000 mgr.y (mgr.24475) 602 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:29 smithi155 bash[20805]: cluster 2024-02-18T01:32:28.973077+0000 mon.a (mon.0) 2691 : cluster [DBG] osdmap e537: 8 total, 8 up, 8 in 2024-02-18T01:32:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:29 smithi155 bash[20805]: audit 2024-02-18T01:32:28.977264+0000 mon.a (mon.0) 2692 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:31.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[27669]: cluster 2024-02-18T01:32:29.912703+0000 mgr.y (mgr.24475) 603 : cluster [DBG] pgmap v926: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[27669]: audit 2024-02-18T01:32:29.969368+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[27669]: cluster 2024-02-18T01:32:29.969523+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[27669]: audit 2024-02-18T01:32:29.993798+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[19734]: cluster 2024-02-18T01:32:29.912703+0000 mgr.y (mgr.24475) 603 : cluster [DBG] pgmap v926: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[19734]: audit 2024-02-18T01:32:29.969368+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[19734]: cluster 2024-02-18T01:32:29.969523+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T01:32:31.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:30 smithi105 bash[19734]: audit 2024-02-18T01:32:29.993798+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:30 smithi155 bash[20805]: cluster 2024-02-18T01:32:29.912703+0000 mgr.y (mgr.24475) 603 : cluster [DBG] pgmap v926: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:30 smithi155 bash[20805]: audit 2024-02-18T01:32:29.969368+0000 mon.a (mon.0) 2693 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:30 smithi155 bash[20805]: cluster 2024-02-18T01:32:29.969523+0000 mon.a (mon.0) 2694 : cluster [DBG] osdmap e538: 8 total, 8 up, 8 in 2024-02-18T01:32:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:30 smithi155 bash[20805]: audit 2024-02-18T01:32:29.993798+0000 mon.a (mon.0) 2695 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:32.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[27669]: audit 2024-02-18T01:32:30.970704+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[27669]: cluster 2024-02-18T01:32:30.970874+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[27669]: audit 2024-02-18T01:32:30.973505+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[27669]: audit 2024-02-18T01:32:31.972422+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[27669]: cluster 2024-02-18T01:32:31.972600+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[19734]: audit 2024-02-18T01:32:30.970704+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[19734]: cluster 2024-02-18T01:32:30.970874+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[19734]: audit 2024-02-18T01:32:30.973505+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:32.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[19734]: audit 2024-02-18T01:32:31.972422+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:32.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:31 smithi105 bash[19734]: cluster 2024-02-18T01:32:31.972600+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[20805]: audit 2024-02-18T01:32:30.970704+0000 mon.a (mon.0) 2696 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[20805]: cluster 2024-02-18T01:32:30.970874+0000 mon.a (mon.0) 2697 : cluster [DBG] osdmap e539: 8 total, 8 up, 8 in 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[20805]: audit 2024-02-18T01:32:30.973505+0000 mon.a (mon.0) 2698 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[20805]: audit 2024-02-18T01:32:31.972422+0000 mon.a (mon.0) 2699 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:31 smithi155 bash[20805]: cluster 2024-02-18T01:32:31.972600+0000 mon.a (mon.0) 2700 : cluster [DBG] osdmap e540: 8 total, 8 up, 8 in 2024-02-18T01:32:33.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:32 smithi105 bash[27669]: cluster 2024-02-18T01:32:31.913551+0000 mgr.y (mgr.24475) 604 : cluster [DBG] pgmap v929: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:33.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:32 smithi105 bash[27669]: audit 2024-02-18T01:32:32.016211+0000 mon.a (mon.0) 2701 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:32 smithi105 bash[19734]: cluster 2024-02-18T01:32:31.913551+0000 mgr.y (mgr.24475) 604 : cluster [DBG] pgmap v929: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:32 smithi105 bash[19734]: audit 2024-02-18T01:32:32.016211+0000 mon.a (mon.0) 2701 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:32 smithi155 bash[20805]: cluster 2024-02-18T01:32:31.913551+0000 mgr.y (mgr.24475) 604 : cluster [DBG] pgmap v929: 204 pgs: 40 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:32 smithi155 bash[20805]: audit 2024-02-18T01:32:32.016211+0000 mon.a (mon.0) 2701 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:34.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[19734]: audit 2024-02-18T01:32:33.000592+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[19734]: cluster 2024-02-18T01:32:33.000684+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T01:32:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[19734]: audit 2024-02-18T01:32:33.001285+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[27669]: audit 2024-02-18T01:32:33.000592+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[27669]: cluster 2024-02-18T01:32:33.000684+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T01:32:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:33 smithi105 bash[27669]: audit 2024-02-18T01:32:33.001285+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:33 smithi155 bash[20805]: audit 2024-02-18T01:32:33.000592+0000 mon.a (mon.0) 2702 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:33 smithi155 bash[20805]: cluster 2024-02-18T01:32:33.000684+0000 mon.a (mon.0) 2703 : cluster [DBG] osdmap e541: 8 total, 8 up, 8 in 2024-02-18T01:32:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:33 smithi155 bash[20805]: audit 2024-02-18T01:32:33.001285+0000 mon.a (mon.0) 2704 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]: dispatch 2024-02-18T01:32:35.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:34 smithi105 bash[19734]: cluster 2024-02-18T01:32:33.914380+0000 mgr.y (mgr.24475) 605 : cluster [DBG] pgmap v932: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[19734]: cluster 2024-02-18T01:32:33.995338+0000 mon.a (mon.0) 2705 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[19734]: audit 2024-02-18T01:32:34.009272+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[19734]: cluster 2024-02-18T01:32:34.009372+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[27669]: cluster 2024-02-18T01:32:33.914380+0000 mgr.y (mgr.24475) 605 : cluster [DBG] pgmap v932: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[27669]: cluster 2024-02-18T01:32:33.995338+0000 mon.a (mon.0) 2705 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[27669]: audit 2024-02-18T01:32:34.009272+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:35.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:35 smithi105 bash[27669]: cluster 2024-02-18T01:32:34.009372+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T01:32:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:35 smithi155 bash[20805]: cluster 2024-02-18T01:32:33.914380+0000 mgr.y (mgr.24475) 605 : cluster [DBG] pgmap v932: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:32:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:35 smithi155 bash[20805]: cluster 2024-02-18T01:32:33.995338+0000 mon.a (mon.0) 2705 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:32:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:35 smithi155 bash[20805]: audit 2024-02-18T01:32:34.009272+0000 mon.a (mon.0) 2706 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-102"}]': finished 2024-02-18T01:32:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:35 smithi155 bash[20805]: cluster 2024-02-18T01:32:34.009372+0000 mon.a (mon.0) 2707 : cluster [DBG] osdmap e542: 8 total, 8 up, 8 in 2024-02-18T01:32:36.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:36 smithi105 bash[19734]: cluster 2024-02-18T01:32:35.011629+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T01:32:36.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:36 smithi105 bash[27669]: cluster 2024-02-18T01:32:35.011629+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T01:32:36.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:36 smithi155 bash[20805]: cluster 2024-02-18T01:32:35.011629+0000 mon.a (mon.0) 2708 : cluster [DBG] osdmap e543: 8 total, 8 up, 8 in 2024-02-18T01:32:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:36] "GET /metrics HTTP/1.1" 200 190176 "" "Prometheus/2.33.4" 2024-02-18T01:32:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[19734]: cluster 2024-02-18T01:32:35.915237+0000 mgr.y (mgr.24475) 606 : cluster [DBG] pgmap v935: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:37.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[19734]: cluster 2024-02-18T01:32:36.027439+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T01:32:37.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[19734]: audit 2024-02-18T01:32:36.035897+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[27669]: cluster 2024-02-18T01:32:35.915237+0000 mgr.y (mgr.24475) 606 : cluster [DBG] pgmap v935: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[27669]: cluster 2024-02-18T01:32:36.027439+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T01:32:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:37 smithi105 bash[27669]: audit 2024-02-18T01:32:36.035897+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:37 smithi155 bash[20805]: cluster 2024-02-18T01:32:35.915237+0000 mgr.y (mgr.24475) 606 : cluster [DBG] pgmap v935: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:37 smithi155 bash[20805]: cluster 2024-02-18T01:32:36.027439+0000 mon.a (mon.0) 2709 : cluster [DBG] osdmap e544: 8 total, 8 up, 8 in 2024-02-18T01:32:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:37 smithi155 bash[20805]: audit 2024-02-18T01:32:36.035897+0000 mon.a (mon.0) 2710 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:38.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[19734]: audit 2024-02-18T01:32:37.029425+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:38.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[19734]: cluster 2024-02-18T01:32:37.029818+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T01:32:38.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[19734]: audit 2024-02-18T01:32:37.052061+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:38.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[27669]: audit 2024-02-18T01:32:37.029425+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:38.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[27669]: cluster 2024-02-18T01:32:37.029818+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T01:32:38.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:38 smithi105 bash[27669]: audit 2024-02-18T01:32:37.052061+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:38 smithi155 bash[20805]: audit 2024-02-18T01:32:37.029425+0000 mon.a (mon.0) 2711 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-104","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:38 smithi155 bash[20805]: cluster 2024-02-18T01:32:37.029818+0000 mon.a (mon.0) 2712 : cluster [DBG] osdmap e545: 8 total, 8 up, 8 in 2024-02-18T01:32:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:38 smithi155 bash[20805]: audit 2024-02-18T01:32:37.052061+0000 mon.a (mon.0) 2713 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: cluster 2024-02-18T01:32:37.917183+0000 mgr.y (mgr.24475) 607 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:32:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: audit 2024-02-18T01:32:38.043320+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: cluster 2024-02-18T01:32:38.043421+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: audit 2024-02-18T01:32:38.044509+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: audit 2024-02-18T01:32:38.081619+0000 mgr.y (mgr.24475) 608 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: audit 2024-02-18T01:32:39.046452+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[19734]: cluster 2024-02-18T01:32:39.046530+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: cluster 2024-02-18T01:32:37.917183+0000 mgr.y (mgr.24475) 607 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: audit 2024-02-18T01:32:38.043320+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: cluster 2024-02-18T01:32:38.043421+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: audit 2024-02-18T01:32:38.044509+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: audit 2024-02-18T01:32:38.081619+0000 mgr.y (mgr.24475) 608 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: audit 2024-02-18T01:32:39.046452+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:39.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:39 smithi105 bash[27669]: cluster 2024-02-18T01:32:39.046530+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: cluster 2024-02-18T01:32:37.917183+0000 mgr.y (mgr.24475) 607 : cluster [DBG] pgmap v938: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: audit 2024-02-18T01:32:38.043320+0000 mon.a (mon.0) 2714 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: cluster 2024-02-18T01:32:38.043421+0000 mon.a (mon.0) 2715 : cluster [DBG] osdmap e546: 8 total, 8 up, 8 in 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: audit 2024-02-18T01:32:38.044509+0000 mon.a (mon.0) 2716 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: audit 2024-02-18T01:32:38.081619+0000 mgr.y (mgr.24475) 608 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: audit 2024-02-18T01:32:39.046452+0000 mon.a (mon.0) 2717 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:39 smithi155 bash[20805]: cluster 2024-02-18T01:32:39.046530+0000 mon.a (mon.0) 2718 : cluster [DBG] osdmap e547: 8 total, 8 up, 8 in 2024-02-18T01:32:40.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:40 smithi105 bash[19734]: audit 2024-02-18T01:32:39.049857+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]: dispatch 2024-02-18T01:32:40.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:40 smithi105 bash[19734]: cluster 2024-02-18T01:32:40.040517+0000 mon.a (mon.0) 2720 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:40.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:40 smithi105 bash[27669]: audit 2024-02-18T01:32:39.049857+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]: dispatch 2024-02-18T01:32:40.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:40 smithi105 bash[27669]: cluster 2024-02-18T01:32:40.040517+0000 mon.a (mon.0) 2720 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:40 smithi155 bash[20805]: audit 2024-02-18T01:32:39.049857+0000 mon.a (mon.0) 2719 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]: dispatch 2024-02-18T01:32:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:40 smithi155 bash[20805]: cluster 2024-02-18T01:32:40.040517+0000 mon.a (mon.0) 2720 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[19734]: cluster 2024-02-18T01:32:39.918006+0000 mgr.y (mgr.24475) 609 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[19734]: audit 2024-02-18T01:32:40.051188+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]': finished 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[19734]: cluster 2024-02-18T01:32:40.051306+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[19734]: audit 2024-02-18T01:32:40.110010+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[27669]: cluster 2024-02-18T01:32:39.918006+0000 mgr.y (mgr.24475) 609 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[27669]: audit 2024-02-18T01:32:40.051188+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]': finished 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[27669]: cluster 2024-02-18T01:32:40.051306+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T01:32:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:41 smithi105 bash[27669]: audit 2024-02-18T01:32:40.110010+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:41 smithi155 bash[20805]: cluster 2024-02-18T01:32:39.918006+0000 mgr.y (mgr.24475) 609 : cluster [DBG] pgmap v941: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:32:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:41 smithi155 bash[20805]: audit 2024-02-18T01:32:40.051188+0000 mon.a (mon.0) 2721 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-104", "mode": "writeback"}]': finished 2024-02-18T01:32:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:41 smithi155 bash[20805]: cluster 2024-02-18T01:32:40.051306+0000 mon.a (mon.0) 2722 : cluster [DBG] osdmap e548: 8 total, 8 up, 8 in 2024-02-18T01:32:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:41 smithi155 bash[20805]: audit 2024-02-18T01:32:40.110010+0000 mon.a (mon.0) 2723 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:32:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:42 smithi155 bash[20805]: audit 2024-02-18T01:32:41.070696+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:42 smithi155 bash[20805]: cluster 2024-02-18T01:32:41.070795+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T01:32:42.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:42 smithi155 bash[20805]: audit 2024-02-18T01:32:41.071843+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[19734]: audit 2024-02-18T01:32:41.070696+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[19734]: cluster 2024-02-18T01:32:41.070795+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T01:32:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[19734]: audit 2024-02-18T01:32:41.071843+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[27669]: audit 2024-02-18T01:32:41.070696+0000 mon.a (mon.0) 2724 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:32:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[27669]: cluster 2024-02-18T01:32:41.070795+0000 mon.a (mon.0) 2725 : cluster [DBG] osdmap e549: 8 total, 8 up, 8 in 2024-02-18T01:32:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:42 smithi105 bash[27669]: audit 2024-02-18T01:32:41.071843+0000 mon.a (mon.0) 2726 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]: dispatch 2024-02-18T01:32:43.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[27669]: cluster 2024-02-18T01:32:41.918526+0000 mgr.y (mgr.24475) 610 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:43.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[27669]: cluster 2024-02-18T01:32:42.066090+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:43.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[27669]: audit 2024-02-18T01:32:42.079993+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:43.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[27669]: cluster 2024-02-18T01:32:42.080116+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T01:32:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[19734]: cluster 2024-02-18T01:32:41.918526+0000 mgr.y (mgr.24475) 610 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[19734]: cluster 2024-02-18T01:32:42.066090+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[19734]: audit 2024-02-18T01:32:42.079993+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:43.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:43 smithi105 bash[19734]: cluster 2024-02-18T01:32:42.080116+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T01:32:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:43 smithi155 bash[20805]: cluster 2024-02-18T01:32:41.918526+0000 mgr.y (mgr.24475) 610 : cluster [DBG] pgmap v944: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:32:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:43 smithi155 bash[20805]: cluster 2024-02-18T01:32:42.066090+0000 mon.a (mon.0) 2727 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:32:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:43 smithi155 bash[20805]: audit 2024-02-18T01:32:42.079993+0000 mon.a (mon.0) 2728 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-104"}]': finished 2024-02-18T01:32:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:43 smithi155 bash[20805]: cluster 2024-02-18T01:32:42.080116+0000 mon.a (mon.0) 2729 : cluster [DBG] osdmap e550: 8 total, 8 up, 8 in 2024-02-18T01:32:44.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:44 smithi105 bash[27669]: cluster 2024-02-18T01:32:43.088035+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T01:32:44.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:44 smithi105 bash[19734]: cluster 2024-02-18T01:32:43.088035+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T01:32:44.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:44 smithi155 bash[20805]: cluster 2024-02-18T01:32:43.088035+0000 mon.a (mon.0) 2730 : cluster [DBG] osdmap e551: 8 total, 8 up, 8 in 2024-02-18T01:32:45.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[27669]: cluster 2024-02-18T01:32:43.919291+0000 mgr.y (mgr.24475) 611 : cluster [DBG] pgmap v947: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:45.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[27669]: cluster 2024-02-18T01:32:44.100698+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T01:32:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[27669]: audit 2024-02-18T01:32:44.102107+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[19734]: cluster 2024-02-18T01:32:43.919291+0000 mgr.y (mgr.24475) 611 : cluster [DBG] pgmap v947: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[19734]: cluster 2024-02-18T01:32:44.100698+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T01:32:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:45 smithi105 bash[19734]: audit 2024-02-18T01:32:44.102107+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:45 smithi155 bash[20805]: cluster 2024-02-18T01:32:43.919291+0000 mgr.y (mgr.24475) 611 : cluster [DBG] pgmap v947: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:45 smithi155 bash[20805]: cluster 2024-02-18T01:32:44.100698+0000 mon.a (mon.0) 2731 : cluster [DBG] osdmap e552: 8 total, 8 up, 8 in 2024-02-18T01:32:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:45 smithi155 bash[20805]: audit 2024-02-18T01:32:44.102107+0000 mon.a (mon.0) 2732 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:32:46.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[19734]: audit 2024-02-18T01:32:45.111122+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:46.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[19734]: cluster 2024-02-18T01:32:45.111356+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T01:32:46.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[19734]: cluster 2024-02-18T01:32:45.920060+0000 mgr.y (mgr.24475) 612 : cluster [DBG] pgmap v950: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[27669]: audit 2024-02-18T01:32:45.111122+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[27669]: cluster 2024-02-18T01:32:45.111356+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T01:32:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[27669]: cluster 2024-02-18T01:32:45.920060+0000 mgr.y (mgr.24475) 612 : cluster [DBG] pgmap v950: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:46 smithi155 bash[20805]: audit 2024-02-18T01:32:45.111122+0000 mon.a (mon.0) 2733 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-106","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:32:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:46 smithi155 bash[20805]: cluster 2024-02-18T01:32:45.111356+0000 mon.a (mon.0) 2734 : cluster [DBG] osdmap e553: 8 total, 8 up, 8 in 2024-02-18T01:32:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:46 smithi155 bash[20805]: cluster 2024-02-18T01:32:45.920060+0000 mgr.y (mgr.24475) 612 : cluster [DBG] pgmap v950: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:46] "GET /metrics HTTP/1.1" 200 190176 "" "Prometheus/2.33.4" 2024-02-18T01:32:47.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:47 smithi105 bash[27669]: cluster 2024-02-18T01:32:46.122470+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T01:32:47.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:47 smithi105 bash[27669]: audit 2024-02-18T01:32:46.144814+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:47 smithi105 bash[19734]: cluster 2024-02-18T01:32:46.122470+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T01:32:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:47 smithi105 bash[19734]: audit 2024-02-18T01:32:46.144814+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:47 smithi155 bash[20805]: cluster 2024-02-18T01:32:46.122470+0000 mon.a (mon.0) 2735 : cluster [DBG] osdmap e554: 8 total, 8 up, 8 in 2024-02-18T01:32:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:47 smithi155 bash[20805]: audit 2024-02-18T01:32:46.144814+0000 mon.a (mon.0) 2736 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:32:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[19734]: audit 2024-02-18T01:32:47.135032+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[19734]: cluster 2024-02-18T01:32:47.135213+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[19734]: audit 2024-02-18T01:32:47.136523+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[19734]: cluster 2024-02-18T01:32:47.922081+0000 mgr.y (mgr.24475) 613 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[19734]: audit 2024-02-18T01:32:48.085869+0000 mgr.y (mgr.24475) 614 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[27669]: audit 2024-02-18T01:32:47.135032+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[27669]: cluster 2024-02-18T01:32:47.135213+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[27669]: audit 2024-02-18T01:32:47.136523+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[27669]: cluster 2024-02-18T01:32:47.922081+0000 mgr.y (mgr.24475) 613 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:48 smithi105 bash[27669]: audit 2024-02-18T01:32:48.085869+0000 mgr.y (mgr.24475) 614 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[20805]: audit 2024-02-18T01:32:47.135032+0000 mon.a (mon.0) 2737 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:32:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[20805]: cluster 2024-02-18T01:32:47.135213+0000 mon.a (mon.0) 2738 : cluster [DBG] osdmap e555: 8 total, 8 up, 8 in 2024-02-18T01:32:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[20805]: audit 2024-02-18T01:32:47.136523+0000 mon.a (mon.0) 2739 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:32:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[20805]: cluster 2024-02-18T01:32:47.922081+0000 mgr.y (mgr.24475) 613 : cluster [DBG] pgmap v953: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:48 smithi155 bash[20805]: audit 2024-02-18T01:32:48.085869+0000 mgr.y (mgr.24475) 614 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:32:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[19734]: audit 2024-02-18T01:32:48.139919+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[19734]: cluster 2024-02-18T01:32:48.140105+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[19734]: audit 2024-02-18T01:32:48.142304+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]: dispatch 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[19734]: cluster 2024-02-18T01:32:49.133339+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[27669]: audit 2024-02-18T01:32:48.139919+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[27669]: cluster 2024-02-18T01:32:48.140105+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[27669]: audit 2024-02-18T01:32:48.142304+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]: dispatch 2024-02-18T01:32:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:49 smithi105 bash[27669]: cluster 2024-02-18T01:32:49.133339+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:49 smithi155 bash[20805]: audit 2024-02-18T01:32:48.139919+0000 mon.a (mon.0) 2740 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:32:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:49 smithi155 bash[20805]: cluster 2024-02-18T01:32:48.140105+0000 mon.a (mon.0) 2741 : cluster [DBG] osdmap e556: 8 total, 8 up, 8 in 2024-02-18T01:32:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:49 smithi155 bash[20805]: audit 2024-02-18T01:32:48.142304+0000 mon.a (mon.0) 2742 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]: dispatch 2024-02-18T01:32:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:49 smithi155 bash[20805]: cluster 2024-02-18T01:32:49.133339+0000 mon.a (mon.0) 2743 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:32:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[19734]: audit 2024-02-18T01:32:49.146525+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]': finished 2024-02-18T01:32:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[19734]: cluster 2024-02-18T01:32:49.146688+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[19734]: audit 2024-02-18T01:32:49.168503+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[19734]: audit 2024-02-18T01:32:49.168760+0000 mgr.y (mgr.24475) 615 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[19734]: cluster 2024-02-18T01:32:49.922911+0000 mgr.y (mgr.24475) 616 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[27669]: audit 2024-02-18T01:32:49.146525+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]': finished 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[27669]: cluster 2024-02-18T01:32:49.146688+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[27669]: audit 2024-02-18T01:32:49.168503+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[27669]: audit 2024-02-18T01:32:49.168760+0000 mgr.y (mgr.24475) 615 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:50 smithi105 bash[27669]: cluster 2024-02-18T01:32:49.922911+0000 mgr.y (mgr.24475) 616 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:50 smithi155 bash[20805]: audit 2024-02-18T01:32:49.146525+0000 mon.a (mon.0) 2744 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-106", "mode": "writeback"}]': finished 2024-02-18T01:32:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:50 smithi155 bash[20805]: cluster 2024-02-18T01:32:49.146688+0000 mon.a (mon.0) 2745 : cluster [DBG] osdmap e557: 8 total, 8 up, 8 in 2024-02-18T01:32:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:50 smithi155 bash[20805]: audit 2024-02-18T01:32:49.168503+0000 mon.a (mon.0) 2746 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:50 smithi155 bash[20805]: audit 2024-02-18T01:32:49.168760+0000 mgr.y (mgr.24475) 615 : audit [DBG] from='mon.0 v1:172.21.15.105:0/828247636' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "237.4"}]: dispatch 2024-02-18T01:32:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:50 smithi155 bash[20805]: cluster 2024-02-18T01:32:49.922911+0000 mgr.y (mgr.24475) 616 : cluster [DBG] pgmap v956: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:32:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:51 smithi105 bash[19734]: cluster 2024-02-18T01:32:50.059777+0000 osd.2 (osd.2) 7 : cluster [DBG] 237.4 scrub starts 2024-02-18T01:32:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:51 smithi105 bash[19734]: cluster 2024-02-18T01:32:50.063573+0000 osd.2 (osd.2) 8 : cluster [DBG] 237.4 scrub ok 2024-02-18T01:32:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:51 smithi105 bash[27669]: cluster 2024-02-18T01:32:50.059777+0000 osd.2 (osd.2) 7 : cluster [DBG] 237.4 scrub starts 2024-02-18T01:32:51.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:51 smithi105 bash[27669]: cluster 2024-02-18T01:32:50.063573+0000 osd.2 (osd.2) 8 : cluster [DBG] 237.4 scrub ok 2024-02-18T01:32:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:51 smithi155 bash[20805]: cluster 2024-02-18T01:32:50.059777+0000 osd.2 (osd.2) 7 : cluster [DBG] 237.4 scrub starts 2024-02-18T01:32:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:51 smithi155 bash[20805]: cluster 2024-02-18T01:32:50.063573+0000 osd.2 (osd.2) 8 : cluster [DBG] 237.4 scrub ok 2024-02-18T01:32:52.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:32:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:32:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:52 smithi105 bash[19734]: audit 2024-02-18T01:32:51.519373+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:52 smithi105 bash[19734]: cluster 2024-02-18T01:32:51.923762+0000 mgr.y (mgr.24475) 617 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:32:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:52 smithi105 bash[27669]: audit 2024-02-18T01:32:51.519373+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:52 smithi105 bash[27669]: cluster 2024-02-18T01:32:51.923762+0000 mgr.y (mgr.24475) 617 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:32:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:52 smithi155 bash[20805]: audit 2024-02-18T01:32:51.519373+0000 mon.a (mon.0) 2747 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:32:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:52 smithi155 bash[20805]: cluster 2024-02-18T01:32:51.923762+0000 mgr.y (mgr.24475) 617 : cluster [DBG] pgmap v957: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:32:55.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:54 smithi105 bash[19734]: cluster 2024-02-18T01:32:53.925872+0000 mgr.y (mgr.24475) 618 : cluster [DBG] pgmap v958: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:32:55.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:54 smithi105 bash[27669]: cluster 2024-02-18T01:32:53.925872+0000 mgr.y (mgr.24475) 618 : cluster [DBG] pgmap v958: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:32:55.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:54 smithi155 bash[20805]: cluster 2024-02-18T01:32:53.925872+0000 mgr.y (mgr.24475) 618 : cluster [DBG] pgmap v958: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-02-18T01:32:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:32:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:32:56] "GET /metrics HTTP/1.1" 200 190306 "" "Prometheus/2.33.4" 2024-02-18T01:32:57.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:56 smithi105 bash[19734]: cluster 2024-02-18T01:32:55.926677+0000 mgr.y (mgr.24475) 619 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-02-18T01:32:57.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:56 smithi105 bash[27669]: cluster 2024-02-18T01:32:55.926677+0000 mgr.y (mgr.24475) 619 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-02-18T01:32:57.244 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:56 smithi155 bash[20805]: cluster 2024-02-18T01:32:55.926677+0000 mgr.y (mgr.24475) 619 : cluster [DBG] pgmap v959: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 1 op/s 2024-02-18T01:32:58.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:32:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:32:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:32:58 smithi105 bash[19734]: cluster 2024-02-18T01:32:57.928770+0000 mgr.y (mgr.24475) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:32:58 smithi105 bash[27669]: cluster 2024-02-18T01:32:57.928770+0000 mgr.y (mgr.24475) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:32:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:32:58 smithi155 bash[20805]: cluster 2024-02-18T01:32:57.928770+0000 mgr.y (mgr.24475) 620 : cluster [DBG] pgmap v960: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:00.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:00 smithi105 bash[27669]: audit 2024-02-18T01:32:58.088984+0000 mgr.y (mgr.24475) 621 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:00 smithi105 bash[19734]: audit 2024-02-18T01:32:58.088984+0000 mgr.y (mgr.24475) 621 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:00 smithi155 bash[20805]: audit 2024-02-18T01:32:58.088984+0000 mgr.y (mgr.24475) 621 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:01 smithi105 bash[19734]: cluster 2024-02-18T01:32:59.929744+0000 mgr.y (mgr.24475) 622 : cluster [DBG] pgmap v961: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:33:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:01 smithi105 bash[27669]: cluster 2024-02-18T01:32:59.929744+0000 mgr.y (mgr.24475) 622 : cluster [DBG] pgmap v961: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:33:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:01 smithi155 bash[20805]: cluster 2024-02-18T01:32:59.929744+0000 mgr.y (mgr.24475) 622 : cluster [DBG] pgmap v961: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:33:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:03 smithi105 bash[19734]: cluster 2024-02-18T01:33:01.930597+0000 mgr.y (mgr.24475) 623 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-18T01:33:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:03 smithi105 bash[27669]: cluster 2024-02-18T01:33:01.930597+0000 mgr.y (mgr.24475) 623 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-18T01:33:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:03 smithi155 bash[20805]: cluster 2024-02-18T01:33:01.930597+0000 mgr.y (mgr.24475) 623 : cluster [DBG] pgmap v962: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1 op/s 2024-02-18T01:33:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:05 smithi105 bash[19734]: cluster 2024-02-18T01:33:03.932791+0000 mgr.y (mgr.24475) 624 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T01:33:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:05 smithi105 bash[27669]: cluster 2024-02-18T01:33:03.932791+0000 mgr.y (mgr.24475) 624 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T01:33:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:05 smithi155 bash[20805]: cluster 2024-02-18T01:33:03.932791+0000 mgr.y (mgr.24475) 624 : cluster [DBG] pgmap v963: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-02-18T01:33:06.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:06 smithi105 bash[19734]: cluster 2024-02-18T01:33:05.019159+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T01:33:06.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:06 smithi105 bash[19734]: audit 2024-02-18T01:33:05.049445+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:06.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:06 smithi105 bash[27669]: cluster 2024-02-18T01:33:05.019159+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T01:33:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:06 smithi105 bash[27669]: audit 2024-02-18T01:33:05.049445+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:06 smithi155 bash[20805]: cluster 2024-02-18T01:33:05.019159+0000 mon.a (mon.0) 2748 : cluster [DBG] osdmap e558: 8 total, 8 up, 8 in 2024-02-18T01:33:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:06 smithi155 bash[20805]: audit 2024-02-18T01:33:05.049445+0000 mon.a (mon.0) 2749 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:06] "GET /metrics HTTP/1.1" 200 190341 "" "Prometheus/2.33.4" 2024-02-18T01:33:07.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: cluster 2024-02-18T01:33:05.933636+0000 mgr.y (mgr.24475) 625 : cluster [DBG] pgmap v965: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:33:07.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: audit 2024-02-18T01:33:06.032756+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: cluster 2024-02-18T01:33:06.032845+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: audit 2024-02-18T01:33:06.036646+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: cluster 2024-02-18T01:33:07.024851+0000 mon.a (mon.0) 2753 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: audit 2024-02-18T01:33:07.033145+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[27669]: cluster 2024-02-18T01:33:07.033369+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: cluster 2024-02-18T01:33:05.933636+0000 mgr.y (mgr.24475) 625 : cluster [DBG] pgmap v965: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: audit 2024-02-18T01:33:06.032756+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: cluster 2024-02-18T01:33:06.032845+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: audit 2024-02-18T01:33:06.036646+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: cluster 2024-02-18T01:33:07.024851+0000 mon.a (mon.0) 2753 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: audit 2024-02-18T01:33:07.033145+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:33:07.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:07 smithi105 bash[19734]: cluster 2024-02-18T01:33:07.033369+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: cluster 2024-02-18T01:33:05.933636+0000 mgr.y (mgr.24475) 625 : cluster [DBG] pgmap v965: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: audit 2024-02-18T01:33:06.032756+0000 mon.a (mon.0) 2750 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: cluster 2024-02-18T01:33:06.032845+0000 mon.a (mon.0) 2751 : cluster [DBG] osdmap e559: 8 total, 8 up, 8 in 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: audit 2024-02-18T01:33:06.036646+0000 mon.a (mon.0) 2752 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]: dispatch 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: cluster 2024-02-18T01:33:07.024851+0000 mon.a (mon.0) 2753 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: audit 2024-02-18T01:33:07.033145+0000 mon.a (mon.0) 2754 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-106"}]': finished 2024-02-18T01:33:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:07 smithi155 bash[20805]: cluster 2024-02-18T01:33:07.033369+0000 mon.a (mon.0) 2755 : cluster [DBG] osdmap e560: 8 total, 8 up, 8 in 2024-02-18T01:33:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:09 smithi105 bash[19734]: cluster 2024-02-18T01:33:07.935623+0000 mgr.y (mgr.24475) 626 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:33:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:09 smithi105 bash[19734]: cluster 2024-02-18T01:33:08.050907+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T01:33:09.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:09 smithi105 bash[27669]: cluster 2024-02-18T01:33:07.935623+0000 mgr.y (mgr.24475) 626 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:33:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:09 smithi105 bash[27669]: cluster 2024-02-18T01:33:08.050907+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T01:33:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:09 smithi155 bash[20805]: cluster 2024-02-18T01:33:07.935623+0000 mgr.y (mgr.24475) 626 : cluster [DBG] pgmap v968: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 0 B/s wr, 3 op/s 2024-02-18T01:33:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:09 smithi155 bash[20805]: cluster 2024-02-18T01:33:08.050907+0000 mon.a (mon.0) 2756 : cluster [DBG] osdmap e561: 8 total, 8 up, 8 in 2024-02-18T01:33:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[19734]: audit 2024-02-18T01:33:08.094877+0000 mgr.y (mgr.24475) 627 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[19734]: cluster 2024-02-18T01:33:09.062222+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T01:33:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[19734]: audit 2024-02-18T01:33:09.063053+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[19734]: audit 2024-02-18T01:33:09.472640+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[19734]: cluster 2024-02-18T01:33:09.473516+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[27669]: audit 2024-02-18T01:33:08.094877+0000 mgr.y (mgr.24475) 627 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[27669]: cluster 2024-02-18T01:33:09.062222+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[27669]: audit 2024-02-18T01:33:09.063053+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[27669]: audit 2024-02-18T01:33:09.472640+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:10 smithi105 bash[27669]: cluster 2024-02-18T01:33:09.473516+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T01:33:10.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:10 smithi155 bash[20805]: audit 2024-02-18T01:33:08.094877+0000 mgr.y (mgr.24475) 627 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:10 smithi155 bash[20805]: cluster 2024-02-18T01:33:09.062222+0000 mon.a (mon.0) 2757 : cluster [DBG] osdmap e562: 8 total, 8 up, 8 in 2024-02-18T01:33:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:10 smithi155 bash[20805]: audit 2024-02-18T01:33:09.063053+0000 mon.a (mon.0) 2758 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:10 smithi155 bash[20805]: audit 2024-02-18T01:33:09.472640+0000 mon.a (mon.0) 2759 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-108","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:10 smithi155 bash[20805]: cluster 2024-02-18T01:33:09.473516+0000 mon.a (mon.0) 2760 : cluster [DBG] osdmap e563: 8 total, 8 up, 8 in 2024-02-18T01:33:11.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[27669]: cluster 2024-02-18T01:33:09.936469+0000 mgr.y (mgr.24475) 628 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 1 op/s 2024-02-18T01:33:11.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[27669]: cluster 2024-02-18T01:33:10.475287+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T01:33:11.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[27669]: audit 2024-02-18T01:33:10.483319+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[19734]: cluster 2024-02-18T01:33:09.936469+0000 mgr.y (mgr.24475) 628 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 1 op/s 2024-02-18T01:33:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[19734]: cluster 2024-02-18T01:33:10.475287+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T01:33:11.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:11 smithi105 bash[19734]: audit 2024-02-18T01:33:10.483319+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:11.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:11 smithi155 bash[20805]: cluster 2024-02-18T01:33:09.936469+0000 mgr.y (mgr.24475) 628 : cluster [DBG] pgmap v972: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 785 B/s rd, 1 op/s 2024-02-18T01:33:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:11 smithi155 bash[20805]: cluster 2024-02-18T01:33:10.475287+0000 mon.a (mon.0) 2761 : cluster [DBG] osdmap e564: 8 total, 8 up, 8 in 2024-02-18T01:33:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:11 smithi155 bash[20805]: audit 2024-02-18T01:33:10.483319+0000 mon.a (mon.0) 2762 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:12.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[19734]: audit 2024-02-18T01:33:11.489987+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:12.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[19734]: cluster 2024-02-18T01:33:11.490076+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[19734]: audit 2024-02-18T01:33:11.490949+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[19734]: cluster 2024-02-18T01:33:11.937261+0000 mgr.y (mgr.24475) 629 : cluster [DBG] pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[27669]: audit 2024-02-18T01:33:11.489987+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[27669]: cluster 2024-02-18T01:33:11.490076+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[27669]: audit 2024-02-18T01:33:11.490949+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:12.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:12 smithi105 bash[27669]: cluster 2024-02-18T01:33:11.937261+0000 mgr.y (mgr.24475) 629 : cluster [DBG] pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:12 smithi155 bash[20805]: audit 2024-02-18T01:33:11.489987+0000 mon.a (mon.0) 2763 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:12 smithi155 bash[20805]: cluster 2024-02-18T01:33:11.490076+0000 mon.a (mon.0) 2764 : cluster [DBG] osdmap e565: 8 total, 8 up, 8 in 2024-02-18T01:33:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:12 smithi155 bash[20805]: audit 2024-02-18T01:33:11.490949+0000 mon.a (mon.0) 2765 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:12.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:12 smithi155 bash[20805]: cluster 2024-02-18T01:33:11.937261+0000 mgr.y (mgr.24475) 629 : cluster [DBG] pgmap v975: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[19734]: audit 2024-02-18T01:33:12.501506+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[19734]: cluster 2024-02-18T01:33:12.501599+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T01:33:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[19734]: audit 2024-02-18T01:33:12.502323+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]: dispatch 2024-02-18T01:33:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[27669]: audit 2024-02-18T01:33:12.501506+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:13.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[27669]: cluster 2024-02-18T01:33:12.501599+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T01:33:13.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:13 smithi105 bash[27669]: audit 2024-02-18T01:33:12.502323+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]: dispatch 2024-02-18T01:33:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:13 smithi155 bash[20805]: audit 2024-02-18T01:33:12.501506+0000 mon.a (mon.0) 2766 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:13 smithi155 bash[20805]: cluster 2024-02-18T01:33:12.501599+0000 mon.a (mon.0) 2767 : cluster [DBG] osdmap e566: 8 total, 8 up, 8 in 2024-02-18T01:33:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:13 smithi155 bash[20805]: audit 2024-02-18T01:33:12.502323+0000 mon.a (mon.0) 2768 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]: dispatch 2024-02-18T01:33:14.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[27669]: cluster 2024-02-18T01:33:13.495307+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:14.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[27669]: audit 2024-02-18T01:33:13.505376+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]': finished 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[27669]: cluster 2024-02-18T01:33:13.505465+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[27669]: cluster 2024-02-18T01:33:13.938164+0000 mgr.y (mgr.24475) 630 : cluster [DBG] pgmap v978: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[27669]: cluster 2024-02-18T01:33:14.506331+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[19734]: cluster 2024-02-18T01:33:13.495307+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[19734]: audit 2024-02-18T01:33:13.505376+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]': finished 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[19734]: cluster 2024-02-18T01:33:13.505465+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[19734]: cluster 2024-02-18T01:33:13.938164+0000 mgr.y (mgr.24475) 630 : cluster [DBG] pgmap v978: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:14.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:14 smithi105 bash[19734]: cluster 2024-02-18T01:33:14.506331+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T01:33:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:14 smithi155 bash[20805]: cluster 2024-02-18T01:33:13.495307+0000 mon.a (mon.0) 2769 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:14 smithi155 bash[20805]: audit 2024-02-18T01:33:13.505376+0000 mon.a (mon.0) 2770 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-108", "mode": "writeback"}]': finished 2024-02-18T01:33:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:14 smithi155 bash[20805]: cluster 2024-02-18T01:33:13.505465+0000 mon.a (mon.0) 2771 : cluster [DBG] osdmap e567: 8 total, 8 up, 8 in 2024-02-18T01:33:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:14 smithi155 bash[20805]: cluster 2024-02-18T01:33:13.938164+0000 mgr.y (mgr.24475) 630 : cluster [DBG] pgmap v978: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:14 smithi155 bash[20805]: cluster 2024-02-18T01:33:14.506331+0000 mon.a (mon.0) 2772 : cluster [DBG] osdmap e568: 8 total, 8 up, 8 in 2024-02-18T01:33:15.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:15 smithi105 bash[19734]: audit 2024-02-18T01:33:14.541429+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:15.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:15 smithi105 bash[27669]: audit 2024-02-18T01:33:14.541429+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:15.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:15 smithi155 bash[20805]: audit 2024-02-18T01:33:14.541429+0000 mon.a (mon.0) 2773 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:16.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[19734]: audit 2024-02-18T01:33:15.533234+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:16.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[19734]: cluster 2024-02-18T01:33:15.533350+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T01:33:16.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[19734]: audit 2024-02-18T01:33:15.534556+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[19734]: cluster 2024-02-18T01:33:15.939035+0000 mgr.y (mgr.24475) 631 : cluster [DBG] pgmap v981: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[27669]: audit 2024-02-18T01:33:15.533234+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[27669]: cluster 2024-02-18T01:33:15.533350+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[27669]: audit 2024-02-18T01:33:15.534556+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[27669]: cluster 2024-02-18T01:33:15.939035+0000 mgr.y (mgr.24475) 631 : cluster [DBG] pgmap v981: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:16.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:16] "GET /metrics HTTP/1.1" 200 190341 "" "Prometheus/2.33.4" 2024-02-18T01:33:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:16 smithi155 bash[20805]: audit 2024-02-18T01:33:15.533234+0000 mon.a (mon.0) 2774 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:16 smithi155 bash[20805]: cluster 2024-02-18T01:33:15.533350+0000 mon.a (mon.0) 2775 : cluster [DBG] osdmap e569: 8 total, 8 up, 8 in 2024-02-18T01:33:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:16 smithi155 bash[20805]: audit 2024-02-18T01:33:15.534556+0000 mon.a (mon.0) 2776 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]: dispatch 2024-02-18T01:33:16.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:16 smithi155 bash[20805]: cluster 2024-02-18T01:33:15.939035+0000 mgr.y (mgr.24475) 631 : cluster [DBG] pgmap v981: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:33:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[19734]: cluster 2024-02-18T01:33:16.528603+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[19734]: audit 2024-02-18T01:33:16.542143+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:18.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[19734]: cluster 2024-02-18T01:33:16.542275+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T01:33:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[27669]: cluster 2024-02-18T01:33:16.528603+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[27669]: audit 2024-02-18T01:33:16.542143+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:18.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:17 smithi105 bash[27669]: cluster 2024-02-18T01:33:16.542275+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T01:33:18.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:17 smithi155 bash[20805]: cluster 2024-02-18T01:33:16.528603+0000 mon.a (mon.0) 2777 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:17 smithi155 bash[20805]: audit 2024-02-18T01:33:16.542143+0000 mon.a (mon.0) 2778 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-108"}]': finished 2024-02-18T01:33:18.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:17 smithi155 bash[20805]: cluster 2024-02-18T01:33:16.542275+0000 mon.a (mon.0) 2779 : cluster [DBG] osdmap e570: 8 total, 8 up, 8 in 2024-02-18T01:33:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:18 smithi155 bash[20805]: cluster 2024-02-18T01:33:17.545474+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T01:33:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:18 smithi155 bash[20805]: cluster 2024-02-18T01:33:17.939787+0000 mgr.y (mgr.24475) 632 : cluster [DBG] pgmap v984: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:18 smithi155 bash[20805]: cluster 2024-02-18T01:33:18.555108+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T01:33:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:18 smithi155 bash[20805]: audit 2024-02-18T01:33:18.556547+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[27669]: cluster 2024-02-18T01:33:17.545474+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T01:33:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[27669]: cluster 2024-02-18T01:33:17.939787+0000 mgr.y (mgr.24475) 632 : cluster [DBG] pgmap v984: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[27669]: cluster 2024-02-18T01:33:18.555108+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T01:33:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[27669]: audit 2024-02-18T01:33:18.556547+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[19734]: cluster 2024-02-18T01:33:17.545474+0000 mon.a (mon.0) 2780 : cluster [DBG] osdmap e571: 8 total, 8 up, 8 in 2024-02-18T01:33:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[19734]: cluster 2024-02-18T01:33:17.939787+0000 mgr.y (mgr.24475) 632 : cluster [DBG] pgmap v984: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[19734]: cluster 2024-02-18T01:33:18.555108+0000 mon.a (mon.0) 2781 : cluster [DBG] osdmap e572: 8 total, 8 up, 8 in 2024-02-18T01:33:19.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:18 smithi105 bash[19734]: audit 2024-02-18T01:33:18.556547+0000 mon.a (mon.0) 2782 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[19734]: audit 2024-02-18T01:33:18.101531+0000 mgr.y (mgr.24475) 633 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[19734]: audit 2024-02-18T01:33:19.476835+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[19734]: cluster 2024-02-18T01:33:19.477021+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T01:33:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[19734]: audit 2024-02-18T01:33:19.493314+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[27669]: audit 2024-02-18T01:33:18.101531+0000 mgr.y (mgr.24475) 633 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[27669]: audit 2024-02-18T01:33:19.476835+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[27669]: cluster 2024-02-18T01:33:19.477021+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T01:33:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:19 smithi105 bash[27669]: audit 2024-02-18T01:33:19.493314+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:19 smithi155 bash[20805]: audit 2024-02-18T01:33:18.101531+0000 mgr.y (mgr.24475) 633 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:19 smithi155 bash[20805]: audit 2024-02-18T01:33:19.476835+0000 mon.a (mon.0) 2783 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-110","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:19 smithi155 bash[20805]: cluster 2024-02-18T01:33:19.477021+0000 mon.a (mon.0) 2784 : cluster [DBG] osdmap e573: 8 total, 8 up, 8 in 2024-02-18T01:33:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:19 smithi155 bash[20805]: audit 2024-02-18T01:33:19.493314+0000 mon.a (mon.0) 2785 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:21.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[19734]: cluster 2024-02-18T01:33:19.940587+0000 mgr.y (mgr.24475) 634 : cluster [DBG] pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:21.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[19734]: audit 2024-02-18T01:33:20.477625+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:21.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[19734]: cluster 2024-02-18T01:33:20.477683+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T01:33:21.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[19734]: audit 2024-02-18T01:33:20.479713+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[27669]: cluster 2024-02-18T01:33:19.940587+0000 mgr.y (mgr.24475) 634 : cluster [DBG] pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[27669]: audit 2024-02-18T01:33:20.477625+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[27669]: cluster 2024-02-18T01:33:20.477683+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T01:33:21.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:20 smithi105 bash[27669]: audit 2024-02-18T01:33:20.479713+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:20 smithi155 bash[20805]: cluster 2024-02-18T01:33:19.940587+0000 mgr.y (mgr.24475) 634 : cluster [DBG] pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:20 smithi155 bash[20805]: audit 2024-02-18T01:33:20.477625+0000 mon.a (mon.0) 2786 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:20 smithi155 bash[20805]: cluster 2024-02-18T01:33:20.477683+0000 mon.a (mon.0) 2787 : cluster [DBG] osdmap e574: 8 total, 8 up, 8 in 2024-02-18T01:33:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:20 smithi155 bash[20805]: audit 2024-02-18T01:33:20.479713+0000 mon.a (mon.0) 2788 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[27669]: audit 2024-02-18T01:33:21.482377+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[27669]: cluster 2024-02-18T01:33:21.482639+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T01:33:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[27669]: audit 2024-02-18T01:33:21.483352+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]: dispatch 2024-02-18T01:33:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[27669]: audit 2024-02-18T01:33:21.517047+0000 mon.a (mon.0) 2792 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[27669]: cluster 2024-02-18T01:33:21.941401+0000 mgr.y (mgr.24475) 635 : cluster [DBG] pgmap v990: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[19734]: audit 2024-02-18T01:33:21.482377+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[19734]: cluster 2024-02-18T01:33:21.482639+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[19734]: audit 2024-02-18T01:33:21.483352+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]: dispatch 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[19734]: audit 2024-02-18T01:33:21.517047+0000 mon.a (mon.0) 2792 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:22.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:22 smithi105 bash[19734]: cluster 2024-02-18T01:33:21.941401+0000 mgr.y (mgr.24475) 635 : cluster [DBG] pgmap v990: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:22 smithi155 bash[20805]: audit 2024-02-18T01:33:21.482377+0000 mon.a (mon.0) 2789 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:22 smithi155 bash[20805]: cluster 2024-02-18T01:33:21.482639+0000 mon.a (mon.0) 2790 : cluster [DBG] osdmap e575: 8 total, 8 up, 8 in 2024-02-18T01:33:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:22 smithi155 bash[20805]: audit 2024-02-18T01:33:21.483352+0000 mon.a (mon.0) 2791 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]: dispatch 2024-02-18T01:33:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:22 smithi155 bash[20805]: audit 2024-02-18T01:33:21.517047+0000 mon.a (mon.0) 2792 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:22 smithi155 bash[20805]: cluster 2024-02-18T01:33:21.941401+0000 mgr.y (mgr.24475) 635 : cluster [DBG] pgmap v990: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:23 smithi155 bash[20805]: cluster 2024-02-18T01:33:22.475502+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:23 smithi155 bash[20805]: audit 2024-02-18T01:33:22.481591+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]': finished 2024-02-18T01:33:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:23 smithi155 bash[20805]: cluster 2024-02-18T01:33:22.481672+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T01:33:23.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:23 smithi155 bash[20805]: audit 2024-02-18T01:33:22.561783+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:23.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[27669]: cluster 2024-02-18T01:33:22.475502+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:23.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[27669]: audit 2024-02-18T01:33:22.481591+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]': finished 2024-02-18T01:33:23.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[27669]: cluster 2024-02-18T01:33:22.481672+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T01:33:23.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[27669]: audit 2024-02-18T01:33:22.561783+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:23.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[19734]: cluster 2024-02-18T01:33:22.475502+0000 mon.a (mon.0) 2793 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:23.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[19734]: audit 2024-02-18T01:33:22.481591+0000 mon.a (mon.0) 2794 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-110", "mode": "writeback"}]': finished 2024-02-18T01:33:23.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[19734]: cluster 2024-02-18T01:33:22.481672+0000 mon.a (mon.0) 2795 : cluster [DBG] osdmap e576: 8 total, 8 up, 8 in 2024-02-18T01:33:23.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:23 smithi105 bash[19734]: audit 2024-02-18T01:33:22.561783+0000 mon.a (mon.0) 2796 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:24.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[19734]: audit 2024-02-18T01:33:23.525016+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:24.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[19734]: cluster 2024-02-18T01:33:23.525117+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T01:33:24.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[19734]: audit 2024-02-18T01:33:23.527059+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:24.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[19734]: cluster 2024-02-18T01:33:23.942266+0000 mgr.y (mgr.24475) 636 : cluster [DBG] pgmap v993: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:24.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[27669]: audit 2024-02-18T01:33:23.525016+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:24.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[27669]: cluster 2024-02-18T01:33:23.525117+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T01:33:24.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[27669]: audit 2024-02-18T01:33:23.527059+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:24.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:24 smithi105 bash[27669]: cluster 2024-02-18T01:33:23.942266+0000 mgr.y (mgr.24475) 636 : cluster [DBG] pgmap v993: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:24.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:24 smithi155 bash[20805]: audit 2024-02-18T01:33:23.525016+0000 mon.a (mon.0) 2797 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:24.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:24 smithi155 bash[20805]: cluster 2024-02-18T01:33:23.525117+0000 mon.a (mon.0) 2798 : cluster [DBG] osdmap e577: 8 total, 8 up, 8 in 2024-02-18T01:33:24.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:24 smithi155 bash[20805]: audit 2024-02-18T01:33:23.527059+0000 mon.a (mon.0) 2799 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]: dispatch 2024-02-18T01:33:24.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:24 smithi155 bash[20805]: cluster 2024-02-18T01:33:23.942266+0000 mgr.y (mgr.24475) 636 : cluster [DBG] pgmap v993: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:25.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[19734]: cluster 2024-02-18T01:33:24.519761+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:25.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[19734]: audit 2024-02-18T01:33:24.529084+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:25.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[19734]: cluster 2024-02-18T01:33:24.529328+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T01:33:25.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[27669]: cluster 2024-02-18T01:33:24.519761+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:25.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[27669]: audit 2024-02-18T01:33:24.529084+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:25.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:25 smithi105 bash[27669]: cluster 2024-02-18T01:33:24.529328+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T01:33:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:25 smithi155 bash[20805]: cluster 2024-02-18T01:33:24.519761+0000 mon.a (mon.0) 2800 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:25 smithi155 bash[20805]: audit 2024-02-18T01:33:24.529084+0000 mon.a (mon.0) 2801 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-110"}]': finished 2024-02-18T01:33:25.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:25 smithi155 bash[20805]: cluster 2024-02-18T01:33:24.529328+0000 mon.a (mon.0) 2802 : cluster [DBG] osdmap e578: 8 total, 8 up, 8 in 2024-02-18T01:33:26.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:26 smithi105 bash[19734]: cluster 2024-02-18T01:33:25.549749+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T01:33:26.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:26 smithi105 bash[19734]: cluster 2024-02-18T01:33:25.942735+0000 mgr.y (mgr.24475) 637 : cluster [DBG] pgmap v996: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:26.990 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:26] "GET /metrics HTTP/1.1" 200 190433 "" "Prometheus/2.33.4" 2024-02-18T01:33:26.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:26 smithi105 bash[27669]: cluster 2024-02-18T01:33:25.549749+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T01:33:26.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:26 smithi105 bash[27669]: cluster 2024-02-18T01:33:25.942735+0000 mgr.y (mgr.24475) 637 : cluster [DBG] pgmap v996: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:26.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:26 smithi155 bash[20805]: cluster 2024-02-18T01:33:25.549749+0000 mon.a (mon.0) 2803 : cluster [DBG] osdmap e579: 8 total, 8 up, 8 in 2024-02-18T01:33:26.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:26 smithi155 bash[20805]: cluster 2024-02-18T01:33:25.942735+0000 mgr.y (mgr.24475) 637 : cluster [DBG] pgmap v996: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:27.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: cluster 2024-02-18T01:33:26.566319+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T01:33:27.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: audit 2024-02-18T01:33:26.571246+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: audit 2024-02-18T01:33:26.783466+0000 mon.a (mon.0) 2806 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: audit 2024-02-18T01:33:27.102777+0000 mon.a (mon.0) 2807 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: audit 2024-02-18T01:33:27.104231+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[19734]: audit 2024-02-18T01:33:27.114108+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: cluster 2024-02-18T01:33:26.566319+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: audit 2024-02-18T01:33:26.571246+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: audit 2024-02-18T01:33:26.783466+0000 mon.a (mon.0) 2806 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: audit 2024-02-18T01:33:27.102777+0000 mon.a (mon.0) 2807 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: audit 2024-02-18T01:33:27.104231+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:33:27.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:27 smithi105 bash[27669]: audit 2024-02-18T01:33:27.114108+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: cluster 2024-02-18T01:33:26.566319+0000 mon.a (mon.0) 2804 : cluster [DBG] osdmap e580: 8 total, 8 up, 8 in 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: audit 2024-02-18T01:33:26.571246+0000 mon.a (mon.0) 2805 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: audit 2024-02-18T01:33:26.783466+0000 mon.a (mon.0) 2806 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: audit 2024-02-18T01:33:27.102777+0000 mon.a (mon.0) 2807 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: audit 2024-02-18T01:33:27.104231+0000 mon.a (mon.0) 2808 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:33:27.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:27 smithi155 bash[20805]: audit 2024-02-18T01:33:27.114108+0000 mon.a (mon.0) 2809 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:33:28.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[19734]: audit 2024-02-18T01:33:27.570297+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[19734]: cluster 2024-02-18T01:33:27.570589+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T01:33:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[19734]: audit 2024-02-18T01:33:27.590889+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:28.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[19734]: cluster 2024-02-18T01:33:27.943562+0000 mgr.y (mgr.24475) 638 : cluster [DBG] pgmap v999: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[27669]: audit 2024-02-18T01:33:27.570297+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[27669]: cluster 2024-02-18T01:33:27.570589+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T01:33:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[27669]: audit 2024-02-18T01:33:27.590889+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:28.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:28 smithi105 bash[27669]: cluster 2024-02-18T01:33:27.943562+0000 mgr.y (mgr.24475) 638 : cluster [DBG] pgmap v999: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:28 smithi155 bash[20805]: audit 2024-02-18T01:33:27.570297+0000 mon.a (mon.0) 2810 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:28 smithi155 bash[20805]: cluster 2024-02-18T01:33:27.570589+0000 mon.a (mon.0) 2811 : cluster [DBG] osdmap e581: 8 total, 8 up, 8 in 2024-02-18T01:33:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:28 smithi155 bash[20805]: audit 2024-02-18T01:33:27.590889+0000 mon.a (mon.0) 2812 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:28.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:28 smithi155 bash[20805]: cluster 2024-02-18T01:33:27.943562+0000 mgr.y (mgr.24475) 638 : cluster [DBG] pgmap v999: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:29.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[19734]: audit 2024-02-18T01:33:28.106025+0000 mgr.y (mgr.24475) 639 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[19734]: audit 2024-02-18T01:33:28.589537+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[19734]: cluster 2024-02-18T01:33:28.589625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[19734]: audit 2024-02-18T01:33:28.591250+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[27669]: audit 2024-02-18T01:33:28.106025+0000 mgr.y (mgr.24475) 639 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[27669]: audit 2024-02-18T01:33:28.589537+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[27669]: cluster 2024-02-18T01:33:28.589625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T01:33:29.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:29 smithi105 bash[27669]: audit 2024-02-18T01:33:28.591250+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:29.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:29 smithi155 bash[20805]: audit 2024-02-18T01:33:28.106025+0000 mgr.y (mgr.24475) 639 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:29.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:29 smithi155 bash[20805]: audit 2024-02-18T01:33:28.589537+0000 mon.a (mon.0) 2813 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:29.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:29 smithi155 bash[20805]: cluster 2024-02-18T01:33:28.589625+0000 mon.a (mon.0) 2814 : cluster [DBG] osdmap e582: 8 total, 8 up, 8 in 2024-02-18T01:33:29.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:29 smithi155 bash[20805]: audit 2024-02-18T01:33:28.591250+0000 mon.a (mon.0) 2815 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:30.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[19734]: audit 2024-02-18T01:33:29.592544+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:30.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[19734]: cluster 2024-02-18T01:33:29.592696+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[19734]: audit 2024-02-18T01:33:29.594331+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]: dispatch 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[19734]: cluster 2024-02-18T01:33:29.944412+0000 mgr.y (mgr.24475) 640 : cluster [DBG] pgmap v1002: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[19734]: cluster 2024-02-18T01:33:30.585805+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[27669]: audit 2024-02-18T01:33:29.592544+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[27669]: cluster 2024-02-18T01:33:29.592696+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[27669]: audit 2024-02-18T01:33:29.594331+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]: dispatch 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[27669]: cluster 2024-02-18T01:33:29.944412+0000 mgr.y (mgr.24475) 640 : cluster [DBG] pgmap v1002: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:30.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:30 smithi105 bash[27669]: cluster 2024-02-18T01:33:30.585805+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:30 smithi155 bash[20805]: audit 2024-02-18T01:33:29.592544+0000 mon.a (mon.0) 2816 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:30 smithi155 bash[20805]: cluster 2024-02-18T01:33:29.592696+0000 mon.a (mon.0) 2817 : cluster [DBG] osdmap e583: 8 total, 8 up, 8 in 2024-02-18T01:33:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:30 smithi155 bash[20805]: audit 2024-02-18T01:33:29.594331+0000 mon.a (mon.0) 2818 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]: dispatch 2024-02-18T01:33:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:30 smithi155 bash[20805]: cluster 2024-02-18T01:33:29.944412+0000 mgr.y (mgr.24475) 640 : cluster [DBG] pgmap v1002: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:30.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:30 smithi155 bash[20805]: cluster 2024-02-18T01:33:30.585805+0000 mon.a (mon.0) 2819 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:31.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:31 smithi155 bash[20805]: audit 2024-02-18T01:33:30.595989+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]': finished 2024-02-18T01:33:31.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:31 smithi155 bash[20805]: cluster 2024-02-18T01:33:30.596074+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T01:33:31.877 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:31 smithi155 bash[20805]: audit 2024-02-18T01:33:30.657266+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:31.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[27669]: audit 2024-02-18T01:33:30.595989+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]': finished 2024-02-18T01:33:31.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[27669]: cluster 2024-02-18T01:33:30.596074+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T01:33:31.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[27669]: audit 2024-02-18T01:33:30.657266+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[19734]: audit 2024-02-18T01:33:30.595989+0000 mon.a (mon.0) 2820 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-112", "mode": "writeback"}]': finished 2024-02-18T01:33:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[19734]: cluster 2024-02-18T01:33:30.596074+0000 mon.a (mon.0) 2821 : cluster [DBG] osdmap e584: 8 total, 8 up, 8 in 2024-02-18T01:33:31.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:31 smithi105 bash[19734]: audit 2024-02-18T01:33:30.657266+0000 mon.a (mon.0) 2822 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[19734]: audit 2024-02-18T01:33:31.614804+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[19734]: cluster 2024-02-18T01:33:31.614908+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T01:33:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[19734]: audit 2024-02-18T01:33:31.616455+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:32.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[19734]: cluster 2024-02-18T01:33:31.945243+0000 mgr.y (mgr.24475) 641 : cluster [DBG] pgmap v1005: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[27669]: audit 2024-02-18T01:33:31.614804+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[27669]: cluster 2024-02-18T01:33:31.614908+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T01:33:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[27669]: audit 2024-02-18T01:33:31.616455+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:32.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:32 smithi105 bash[27669]: cluster 2024-02-18T01:33:31.945243+0000 mgr.y (mgr.24475) 641 : cluster [DBG] pgmap v1005: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:32 smithi155 bash[20805]: audit 2024-02-18T01:33:31.614804+0000 mon.a (mon.0) 2823 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:32 smithi155 bash[20805]: cluster 2024-02-18T01:33:31.614908+0000 mon.a (mon.0) 2824 : cluster [DBG] osdmap e585: 8 total, 8 up, 8 in 2024-02-18T01:33:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:32 smithi155 bash[20805]: audit 2024-02-18T01:33:31.616455+0000 mon.a (mon.0) 2825 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]: dispatch 2024-02-18T01:33:32.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:32 smithi155 bash[20805]: cluster 2024-02-18T01:33:31.945243+0000 mgr.y (mgr.24475) 641 : cluster [DBG] pgmap v1005: 204 pgs: 32 creating+peering, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:33.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[27669]: cluster 2024-02-18T01:33:32.609799+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:33.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[27669]: audit 2024-02-18T01:33:32.621621+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:33.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[27669]: cluster 2024-02-18T01:33:32.621849+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T01:33:33.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[19734]: cluster 2024-02-18T01:33:32.609799+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:33.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[19734]: audit 2024-02-18T01:33:32.621621+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:33.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:33 smithi105 bash[19734]: cluster 2024-02-18T01:33:32.621849+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T01:33:33.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:33 smithi155 bash[20805]: cluster 2024-02-18T01:33:32.609799+0000 mon.a (mon.0) 2826 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:33.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:33 smithi155 bash[20805]: audit 2024-02-18T01:33:32.621621+0000 mon.a (mon.0) 2827 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-112"}]': finished 2024-02-18T01:33:33.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:33 smithi155 bash[20805]: cluster 2024-02-18T01:33:32.621849+0000 mon.a (mon.0) 2828 : cluster [DBG] osdmap e586: 8 total, 8 up, 8 in 2024-02-18T01:33:34.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:34 smithi105 bash[27669]: cluster 2024-02-18T01:33:33.645059+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T01:33:34.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:34 smithi105 bash[27669]: cluster 2024-02-18T01:33:33.945821+0000 mgr.y (mgr.24475) 642 : cluster [DBG] pgmap v1008: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:34 smithi105 bash[19734]: cluster 2024-02-18T01:33:33.645059+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T01:33:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:34 smithi105 bash[19734]: cluster 2024-02-18T01:33:33.945821+0000 mgr.y (mgr.24475) 642 : cluster [DBG] pgmap v1008: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:34 smithi155 bash[20805]: cluster 2024-02-18T01:33:33.645059+0000 mon.a (mon.0) 2829 : cluster [DBG] osdmap e587: 8 total, 8 up, 8 in 2024-02-18T01:33:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:34 smithi155 bash[20805]: cluster 2024-02-18T01:33:33.945821+0000 mgr.y (mgr.24475) 642 : cluster [DBG] pgmap v1008: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:35 smithi105 bash[27669]: cluster 2024-02-18T01:33:34.649105+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T01:33:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:35 smithi105 bash[27669]: audit 2024-02-18T01:33:34.654801+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:35.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:35 smithi105 bash[19734]: cluster 2024-02-18T01:33:34.649105+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T01:33:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:35 smithi105 bash[19734]: audit 2024-02-18T01:33:34.654801+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:35 smithi155 bash[20805]: cluster 2024-02-18T01:33:34.649105+0000 mon.a (mon.0) 2830 : cluster [DBG] osdmap e588: 8 total, 8 up, 8 in 2024-02-18T01:33:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:35 smithi155 bash[20805]: audit 2024-02-18T01:33:34.654801+0000 mon.a (mon.0) 2831 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:36] "GET /metrics HTTP/1.1" 200 190551 "" "Prometheus/2.33.4" 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[19734]: audit 2024-02-18T01:33:35.653638+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[19734]: cluster 2024-02-18T01:33:35.653806+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[19734]: cluster 2024-02-18T01:33:35.946594+0000 mgr.y (mgr.24475) 643 : cluster [DBG] pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[27669]: audit 2024-02-18T01:33:35.653638+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[27669]: cluster 2024-02-18T01:33:35.653806+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T01:33:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:36 smithi105 bash[27669]: cluster 2024-02-18T01:33:35.946594+0000 mgr.y (mgr.24475) 643 : cluster [DBG] pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:36 smithi155 bash[20805]: audit 2024-02-18T01:33:35.653638+0000 mon.a (mon.0) 2832 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:36 smithi155 bash[20805]: cluster 2024-02-18T01:33:35.653806+0000 mon.a (mon.0) 2833 : cluster [DBG] osdmap e589: 8 total, 8 up, 8 in 2024-02-18T01:33:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:36 smithi155 bash[20805]: cluster 2024-02-18T01:33:35.946594+0000 mgr.y (mgr.24475) 643 : cluster [DBG] pgmap v1011: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:33:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:37 smithi105 bash[19734]: cluster 2024-02-18T01:33:36.677042+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T01:33:37.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:37 smithi105 bash[19734]: audit 2024-02-18T01:33:36.694964+0000 mon.a (mon.0) 2835 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:37.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:37 smithi105 bash[27669]: cluster 2024-02-18T01:33:36.677042+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T01:33:37.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:37 smithi105 bash[27669]: audit 2024-02-18T01:33:36.694964+0000 mon.a (mon.0) 2835 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:37 smithi155 bash[20805]: cluster 2024-02-18T01:33:36.677042+0000 mon.a (mon.0) 2834 : cluster [DBG] osdmap e590: 8 total, 8 up, 8 in 2024-02-18T01:33:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:37 smithi155 bash[20805]: audit 2024-02-18T01:33:36.694964+0000 mon.a (mon.0) 2835 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[19734]: audit 2024-02-18T01:33:37.682364+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[19734]: cluster 2024-02-18T01:33:37.682610+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T01:33:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[19734]: audit 2024-02-18T01:33:37.683605+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[19734]: cluster 2024-02-18T01:33:37.947446+0000 mgr.y (mgr.24475) 644 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[27669]: audit 2024-02-18T01:33:37.682364+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[27669]: cluster 2024-02-18T01:33:37.682610+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T01:33:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[27669]: audit 2024-02-18T01:33:37.683605+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:38.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:38 smithi105 bash[27669]: cluster 2024-02-18T01:33:37.947446+0000 mgr.y (mgr.24475) 644 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:38 smithi155 bash[20805]: audit 2024-02-18T01:33:37.682364+0000 mon.a (mon.0) 2836 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:38 smithi155 bash[20805]: cluster 2024-02-18T01:33:37.682610+0000 mon.a (mon.0) 2837 : cluster [DBG] osdmap e591: 8 total, 8 up, 8 in 2024-02-18T01:33:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:38 smithi155 bash[20805]: audit 2024-02-18T01:33:37.683605+0000 mon.a (mon.0) 2838 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:38 smithi155 bash[20805]: cluster 2024-02-18T01:33:37.947446+0000 mgr.y (mgr.24475) 644 : cluster [DBG] pgmap v1014: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[27669]: audit 2024-02-18T01:33:38.113680+0000 mgr.y (mgr.24475) 645 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[27669]: audit 2024-02-18T01:33:38.683147+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[27669]: cluster 2024-02-18T01:33:38.683328+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T01:33:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[27669]: audit 2024-02-18T01:33:38.683949+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]: dispatch 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[27669]: cluster 2024-02-18T01:33:39.677345+0000 mon.a (mon.0) 2842 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[19734]: audit 2024-02-18T01:33:38.113680+0000 mgr.y (mgr.24475) 645 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[19734]: audit 2024-02-18T01:33:38.683147+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[19734]: cluster 2024-02-18T01:33:38.683328+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[19734]: audit 2024-02-18T01:33:38.683949+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]: dispatch 2024-02-18T01:33:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:39 smithi105 bash[19734]: cluster 2024-02-18T01:33:39.677345+0000 mon.a (mon.0) 2842 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:39 smithi155 bash[20805]: audit 2024-02-18T01:33:38.113680+0000 mgr.y (mgr.24475) 645 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:39 smithi155 bash[20805]: audit 2024-02-18T01:33:38.683147+0000 mon.a (mon.0) 2839 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:39 smithi155 bash[20805]: cluster 2024-02-18T01:33:38.683328+0000 mon.a (mon.0) 2840 : cluster [DBG] osdmap e592: 8 total, 8 up, 8 in 2024-02-18T01:33:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:39 smithi155 bash[20805]: audit 2024-02-18T01:33:38.683949+0000 mon.a (mon.0) 2841 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]: dispatch 2024-02-18T01:33:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:39 smithi155 bash[20805]: cluster 2024-02-18T01:33:39.677345+0000 mon.a (mon.0) 2842 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[19734]: audit 2024-02-18T01:33:39.688828+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]': finished 2024-02-18T01:33:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[19734]: cluster 2024-02-18T01:33:39.688934+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T01:33:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[19734]: cluster 2024-02-18T01:33:39.948272+0000 mgr.y (mgr.24475) 646 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[27669]: audit 2024-02-18T01:33:39.688828+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]': finished 2024-02-18T01:33:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[27669]: cluster 2024-02-18T01:33:39.688934+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T01:33:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:40 smithi105 bash[27669]: cluster 2024-02-18T01:33:39.948272+0000 mgr.y (mgr.24475) 646 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:40 smithi155 bash[20805]: audit 2024-02-18T01:33:39.688828+0000 mon.a (mon.0) 2843 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-114", "mode": "writeback"}]': finished 2024-02-18T01:33:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:40 smithi155 bash[20805]: cluster 2024-02-18T01:33:39.688934+0000 mon.a (mon.0) 2844 : cluster [DBG] osdmap e593: 8 total, 8 up, 8 in 2024-02-18T01:33:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:40 smithi155 bash[20805]: cluster 2024-02-18T01:33:39.948272+0000 mgr.y (mgr.24475) 646 : cluster [DBG] pgmap v1017: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-02-18T01:33:41.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:41 smithi105 bash[27669]: cluster 2024-02-18T01:33:40.710821+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T01:33:41.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:41 smithi105 bash[27669]: audit 2024-02-18T01:33:40.739068+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:41.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:41 smithi105 bash[19734]: cluster 2024-02-18T01:33:40.710821+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T01:33:41.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:41 smithi105 bash[19734]: audit 2024-02-18T01:33:40.739068+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:41.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:41.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:41 smithi155 bash[20805]: cluster 2024-02-18T01:33:40.710821+0000 mon.a (mon.0) 2845 : cluster [DBG] osdmap e594: 8 total, 8 up, 8 in 2024-02-18T01:33:41.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:41 smithi155 bash[20805]: audit 2024-02-18T01:33:40.739068+0000 mon.a (mon.0) 2846 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[27669]: audit 2024-02-18T01:33:41.717317+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[27669]: cluster 2024-02-18T01:33:41.717388+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T01:33:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[27669]: audit 2024-02-18T01:33:41.718119+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:42.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[27669]: cluster 2024-02-18T01:33:41.949088+0000 mgr.y (mgr.24475) 647 : cluster [DBG] pgmap v1020: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[19734]: audit 2024-02-18T01:33:41.717317+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[19734]: cluster 2024-02-18T01:33:41.717388+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T01:33:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[19734]: audit 2024-02-18T01:33:41.718119+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:42 smithi105 bash[19734]: cluster 2024-02-18T01:33:41.949088+0000 mgr.y (mgr.24475) 647 : cluster [DBG] pgmap v1020: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:42 smithi155 bash[20805]: audit 2024-02-18T01:33:41.717317+0000 mon.a (mon.0) 2847 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:42 smithi155 bash[20805]: cluster 2024-02-18T01:33:41.717388+0000 mon.a (mon.0) 2848 : cluster [DBG] osdmap e595: 8 total, 8 up, 8 in 2024-02-18T01:33:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:42 smithi155 bash[20805]: audit 2024-02-18T01:33:41.718119+0000 mon.a (mon.0) 2849 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]: dispatch 2024-02-18T01:33:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:42 smithi155 bash[20805]: cluster 2024-02-18T01:33:41.949088+0000 mgr.y (mgr.24475) 647 : cluster [DBG] pgmap v1020: 204 pgs: 204 active+clean; 5.8 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:43.493 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:33:43 smithi155 bash[46955]: t=2024-02-18T01:33:43+0000 lvl=info msg="Database locked, sleeping then retrying" logger=sqlstore error="database is locked" retry=0 2024-02-18T01:33:43.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[27669]: cluster 2024-02-18T01:33:42.714989+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:43.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[27669]: audit 2024-02-18T01:33:42.731153+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:43.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[27669]: cluster 2024-02-18T01:33:42.731302+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T01:33:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[19734]: cluster 2024-02-18T01:33:42.714989+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[19734]: audit 2024-02-18T01:33:42.731153+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:43 smithi105 bash[19734]: cluster 2024-02-18T01:33:42.731302+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T01:33:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:43 smithi155 bash[20805]: cluster 2024-02-18T01:33:42.714989+0000 mon.a (mon.0) 2850 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:43 smithi155 bash[20805]: audit 2024-02-18T01:33:42.731153+0000 mon.a (mon.0) 2851 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-114"}]': finished 2024-02-18T01:33:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:43 smithi155 bash[20805]: cluster 2024-02-18T01:33:42.731302+0000 mon.a (mon.0) 2852 : cluster [DBG] osdmap e596: 8 total, 8 up, 8 in 2024-02-18T01:33:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:44 smithi155 bash[20805]: cluster 2024-02-18T01:33:43.733270+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T01:33:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:44 smithi155 bash[20805]: cluster 2024-02-18T01:33:43.949764+0000 mgr.y (mgr.24475) 648 : cluster [DBG] pgmap v1023: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:45.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:44 smithi105 bash[27669]: cluster 2024-02-18T01:33:43.733270+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T01:33:45.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:44 smithi105 bash[27669]: cluster 2024-02-18T01:33:43.949764+0000 mgr.y (mgr.24475) 648 : cluster [DBG] pgmap v1023: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:45.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:44 smithi105 bash[19734]: cluster 2024-02-18T01:33:43.733270+0000 mon.a (mon.0) 2853 : cluster [DBG] osdmap e597: 8 total, 8 up, 8 in 2024-02-18T01:33:45.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:44 smithi105 bash[19734]: cluster 2024-02-18T01:33:43.949764+0000 mgr.y (mgr.24475) 648 : cluster [DBG] pgmap v1023: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:46.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:45 smithi105 bash[19734]: cluster 2024-02-18T01:33:44.745776+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T01:33:46.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:45 smithi105 bash[19734]: audit 2024-02-18T01:33:44.753374+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:46.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:45 smithi105 bash[27669]: cluster 2024-02-18T01:33:44.745776+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T01:33:46.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:45 smithi105 bash[27669]: audit 2024-02-18T01:33:44.753374+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:46.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:45 smithi155 bash[20805]: cluster 2024-02-18T01:33:44.745776+0000 mon.a (mon.0) 2854 : cluster [DBG] osdmap e598: 8 total, 8 up, 8 in 2024-02-18T01:33:46.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:45 smithi155 bash[20805]: audit 2024-02-18T01:33:44.753374+0000 mon.a (mon.0) 2855 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:46] "GET /metrics HTTP/1.1" 200 190551 "" "Prometheus/2.33.4" 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[27669]: audit 2024-02-18T01:33:45.755255+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[27669]: cluster 2024-02-18T01:33:45.755463+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[27669]: audit 2024-02-18T01:33:45.760455+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[27669]: cluster 2024-02-18T01:33:45.950569+0000 mgr.y (mgr.24475) 649 : cluster [DBG] pgmap v1026: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[19734]: audit 2024-02-18T01:33:45.755255+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[19734]: cluster 2024-02-18T01:33:45.755463+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T01:33:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[19734]: audit 2024-02-18T01:33:45.760455+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:46.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:46 smithi105 bash[19734]: cluster 2024-02-18T01:33:45.950569+0000 mgr.y (mgr.24475) 649 : cluster [DBG] pgmap v1026: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:47.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:46 smithi155 bash[20805]: audit 2024-02-18T01:33:45.755255+0000 mon.a (mon.0) 2856 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:47.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:46 smithi155 bash[20805]: cluster 2024-02-18T01:33:45.755463+0000 mon.a (mon.0) 2857 : cluster [DBG] osdmap e599: 8 total, 8 up, 8 in 2024-02-18T01:33:47.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:46 smithi155 bash[20805]: audit 2024-02-18T01:33:45.760455+0000 mon.a (mon.0) 2858 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:47.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:46 smithi155 bash[20805]: cluster 2024-02-18T01:33:45.950569+0000 mgr.y (mgr.24475) 649 : cluster [DBG] pgmap v1026: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:48.121 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:47 smithi155 bash[20805]: audit 2024-02-18T01:33:46.759335+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:47 smithi155 bash[20805]: cluster 2024-02-18T01:33:46.759481+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T01:33:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:47 smithi155 bash[20805]: audit 2024-02-18T01:33:46.761290+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:47 smithi155 bash[20805]: audit 2024-02-18T01:33:47.760925+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:48.122 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:47 smithi155 bash[20805]: cluster 2024-02-18T01:33:47.761212+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T01:33:48.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[19734]: audit 2024-02-18T01:33:46.759335+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:48.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[19734]: cluster 2024-02-18T01:33:46.759481+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[19734]: audit 2024-02-18T01:33:46.761290+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[19734]: audit 2024-02-18T01:33:47.760925+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[19734]: cluster 2024-02-18T01:33:47.761212+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[27669]: audit 2024-02-18T01:33:46.759335+0000 mon.a (mon.0) 2859 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[27669]: cluster 2024-02-18T01:33:46.759481+0000 mon.a (mon.0) 2860 : cluster [DBG] osdmap e600: 8 total, 8 up, 8 in 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[27669]: audit 2024-02-18T01:33:46.761290+0000 mon.a (mon.0) 2861 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[27669]: audit 2024-02-18T01:33:47.760925+0000 mon.a (mon.0) 2862 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:48.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:47 smithi105 bash[27669]: cluster 2024-02-18T01:33:47.761212+0000 mon.a (mon.0) 2863 : cluster [DBG] osdmap e601: 8 total, 8 up, 8 in 2024-02-18T01:33:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:49.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[19734]: audit 2024-02-18T01:33:47.762350+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]: dispatch 2024-02-18T01:33:49.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[19734]: cluster 2024-02-18T01:33:47.951331+0000 mgr.y (mgr.24475) 650 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:49.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[19734]: cluster 2024-02-18T01:33:48.755142+0000 mon.a (mon.0) 2865 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[19734]: audit 2024-02-18T01:33:48.763343+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]': finished 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[19734]: cluster 2024-02-18T01:33:48.763512+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[27669]: audit 2024-02-18T01:33:47.762350+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]: dispatch 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[27669]: cluster 2024-02-18T01:33:47.951331+0000 mgr.y (mgr.24475) 650 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[27669]: cluster 2024-02-18T01:33:48.755142+0000 mon.a (mon.0) 2865 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[27669]: audit 2024-02-18T01:33:48.763343+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]': finished 2024-02-18T01:33:49.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:48 smithi105 bash[27669]: cluster 2024-02-18T01:33:48.763512+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T01:33:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[20805]: audit 2024-02-18T01:33:47.762350+0000 mon.a (mon.0) 2864 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]: dispatch 2024-02-18T01:33:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[20805]: cluster 2024-02-18T01:33:47.951331+0000 mgr.y (mgr.24475) 650 : cluster [DBG] pgmap v1029: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[20805]: cluster 2024-02-18T01:33:48.755142+0000 mon.a (mon.0) 2865 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[20805]: audit 2024-02-18T01:33:48.763343+0000 mon.a (mon.0) 2866 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-116", "mode": "writeback"}]': finished 2024-02-18T01:33:49.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:48 smithi155 bash[20805]: cluster 2024-02-18T01:33:48.763512+0000 mon.a (mon.0) 2867 : cluster [DBG] osdmap e602: 8 total, 8 up, 8 in 2024-02-18T01:33:50.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[27669]: audit 2024-02-18T01:33:48.120685+0000 mgr.y (mgr.24475) 651 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:50.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[27669]: audit 2024-02-18T01:33:48.847956+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:50.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[27669]: audit 2024-02-18T01:33:49.486837+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[27669]: cluster 2024-02-18T01:33:49.487109+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[27669]: audit 2024-02-18T01:33:49.489643+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[19734]: audit 2024-02-18T01:33:48.120685+0000 mgr.y (mgr.24475) 651 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[19734]: audit 2024-02-18T01:33:48.847956+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[19734]: audit 2024-02-18T01:33:49.486837+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[19734]: cluster 2024-02-18T01:33:49.487109+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T01:33:50.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:49 smithi105 bash[19734]: audit 2024-02-18T01:33:49.489643+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:50.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:49 smithi155 bash[20805]: audit 2024-02-18T01:33:48.120685+0000 mgr.y (mgr.24475) 651 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:50.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:49 smithi155 bash[20805]: audit 2024-02-18T01:33:48.847956+0000 mon.a (mon.0) 2868 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:33:50.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:49 smithi155 bash[20805]: audit 2024-02-18T01:33:49.486837+0000 mon.a (mon.0) 2869 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:33:50.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:49 smithi155 bash[20805]: cluster 2024-02-18T01:33:49.487109+0000 mon.a (mon.0) 2870 : cluster [DBG] osdmap e603: 8 total, 8 up, 8 in 2024-02-18T01:33:50.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:49 smithi155 bash[20805]: audit 2024-02-18T01:33:49.489643+0000 mon.a (mon.0) 2871 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]: dispatch 2024-02-18T01:33:51.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[19734]: cluster 2024-02-18T01:33:49.951830+0000 mgr.y (mgr.24475) 652 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:51.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[19734]: cluster 2024-02-18T01:33:50.479579+0000 mon.a (mon.0) 2872 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:51.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[19734]: audit 2024-02-18T01:33:50.488261+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:51.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[19734]: cluster 2024-02-18T01:33:50.488433+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T01:33:51.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[27669]: cluster 2024-02-18T01:33:49.951830+0000 mgr.y (mgr.24475) 652 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:51.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[27669]: cluster 2024-02-18T01:33:50.479579+0000 mon.a (mon.0) 2872 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:51.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[27669]: audit 2024-02-18T01:33:50.488261+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:51.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:50 smithi105 bash[27669]: cluster 2024-02-18T01:33:50.488433+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T01:33:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:50 smithi155 bash[20805]: cluster 2024-02-18T01:33:49.951830+0000 mgr.y (mgr.24475) 652 : cluster [DBG] pgmap v1032: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:33:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:50 smithi155 bash[20805]: cluster 2024-02-18T01:33:50.479579+0000 mon.a (mon.0) 2872 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:33:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:50 smithi155 bash[20805]: audit 2024-02-18T01:33:50.488261+0000 mon.a (mon.0) 2873 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-116"}]': finished 2024-02-18T01:33:51.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:50 smithi155 bash[20805]: cluster 2024-02-18T01:33:50.488433+0000 mon.a (mon.0) 2874 : cluster [DBG] osdmap e604: 8 total, 8 up, 8 in 2024-02-18T01:33:52.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:33:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:33:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:52 smithi105 bash[19734]: cluster 2024-02-18T01:33:51.490601+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T01:33:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:52 smithi105 bash[19734]: cluster 2024-02-18T01:33:51.952562+0000 mgr.y (mgr.24475) 653 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:52.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:52 smithi105 bash[27669]: cluster 2024-02-18T01:33:51.490601+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T01:33:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:52 smithi105 bash[27669]: cluster 2024-02-18T01:33:51.952562+0000 mgr.y (mgr.24475) 653 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:52 smithi155 bash[20805]: cluster 2024-02-18T01:33:51.490601+0000 mon.a (mon.0) 2875 : cluster [DBG] osdmap e605: 8 total, 8 up, 8 in 2024-02-18T01:33:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:52 smithi155 bash[20805]: cluster 2024-02-18T01:33:51.952562+0000 mgr.y (mgr.24475) 653 : cluster [DBG] pgmap v1035: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:33:53.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:53 smithi105 bash[19734]: cluster 2024-02-18T01:33:52.502403+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T01:33:53.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:53 smithi105 bash[19734]: audit 2024-02-18T01:33:52.504399+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:53.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:53 smithi105 bash[27669]: cluster 2024-02-18T01:33:52.502403+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T01:33:53.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:53 smithi105 bash[27669]: audit 2024-02-18T01:33:52.504399+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:53 smithi155 bash[20805]: cluster 2024-02-18T01:33:52.502403+0000 mon.a (mon.0) 2876 : cluster [DBG] osdmap e606: 8 total, 8 up, 8 in 2024-02-18T01:33:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:53 smithi155 bash[20805]: audit 2024-02-18T01:33:52.504399+0000 mon.a (mon.0) 2877 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:33:54.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: audit 2024-02-18T01:33:53.506709+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: cluster 2024-02-18T01:33:53.507162+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: audit 2024-02-18T01:33:53.522224+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: cluster 2024-02-18T01:33:53.953400+0000 mgr.y (mgr.24475) 654 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: audit 2024-02-18T01:33:54.507907+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: cluster 2024-02-18T01:33:54.507987+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[19734]: audit 2024-02-18T01:33:54.510513+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: audit 2024-02-18T01:33:53.506709+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: cluster 2024-02-18T01:33:53.507162+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: audit 2024-02-18T01:33:53.522224+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: cluster 2024-02-18T01:33:53.953400+0000 mgr.y (mgr.24475) 654 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: audit 2024-02-18T01:33:54.507907+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: cluster 2024-02-18T01:33:54.507987+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T01:33:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:54 smithi105 bash[27669]: audit 2024-02-18T01:33:54.510513+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: audit 2024-02-18T01:33:53.506709+0000 mon.a (mon.0) 2878 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: cluster 2024-02-18T01:33:53.507162+0000 mon.a (mon.0) 2879 : cluster [DBG] osdmap e607: 8 total, 8 up, 8 in 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: audit 2024-02-18T01:33:53.522224+0000 mon.a (mon.0) 2880 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: cluster 2024-02-18T01:33:53.953400+0000 mgr.y (mgr.24475) 654 : cluster [DBG] pgmap v1038: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: audit 2024-02-18T01:33:54.507907+0000 mon.a (mon.0) 2881 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: cluster 2024-02-18T01:33:54.507987+0000 mon.a (mon.0) 2882 : cluster [DBG] osdmap e608: 8 total, 8 up, 8 in 2024-02-18T01:33:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:54 smithi155 bash[20805]: audit 2024-02-18T01:33:54.510513+0000 mon.a (mon.0) 2883 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:33:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:33:56] "GET /metrics HTTP/1.1" 200 188145 "" "Prometheus/2.33.4" 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[19734]: audit 2024-02-18T01:33:55.512093+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[19734]: cluster 2024-02-18T01:33:55.512220+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[19734]: audit 2024-02-18T01:33:55.514054+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]: dispatch 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[19734]: cluster 2024-02-18T01:33:55.954289+0000 mgr.y (mgr.24475) 655 : cluster [DBG] pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[27669]: audit 2024-02-18T01:33:55.512093+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[27669]: cluster 2024-02-18T01:33:55.512220+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T01:33:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[27669]: audit 2024-02-18T01:33:55.514054+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]: dispatch 2024-02-18T01:33:56.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:56 smithi105 bash[27669]: cluster 2024-02-18T01:33:55.954289+0000 mgr.y (mgr.24475) 655 : cluster [DBG] pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:56 smithi155 bash[20805]: audit 2024-02-18T01:33:55.512093+0000 mon.a (mon.0) 2884 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:33:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:56 smithi155 bash[20805]: cluster 2024-02-18T01:33:55.512220+0000 mon.a (mon.0) 2885 : cluster [DBG] osdmap e609: 8 total, 8 up, 8 in 2024-02-18T01:33:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:56 smithi155 bash[20805]: audit 2024-02-18T01:33:55.514054+0000 mon.a (mon.0) 2886 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]: dispatch 2024-02-18T01:33:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:56 smithi155 bash[20805]: cluster 2024-02-18T01:33:55.954289+0000 mgr.y (mgr.24475) 655 : cluster [DBG] pgmap v1041: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:33:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[19734]: cluster 2024-02-18T01:33:56.505836+0000 mon.a (mon.0) 2887 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[19734]: audit 2024-02-18T01:33:56.518666+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]': finished 2024-02-18T01:33:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[19734]: cluster 2024-02-18T01:33:56.518779+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T01:33:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[27669]: cluster 2024-02-18T01:33:56.505836+0000 mon.a (mon.0) 2887 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[27669]: audit 2024-02-18T01:33:56.518666+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]': finished 2024-02-18T01:33:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:57 smithi105 bash[27669]: cluster 2024-02-18T01:33:56.518779+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T01:33:57.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:57 smithi155 bash[20805]: cluster 2024-02-18T01:33:56.505836+0000 mon.a (mon.0) 2887 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:33:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:57 smithi155 bash[20805]: audit 2024-02-18T01:33:56.518666+0000 mon.a (mon.0) 2888 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-118", "mode": "writeback"}]': finished 2024-02-18T01:33:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:57 smithi155 bash[20805]: cluster 2024-02-18T01:33:56.518779+0000 mon.a (mon.0) 2889 : cluster [DBG] osdmap e610: 8 total, 8 up, 8 in 2024-02-18T01:33:58.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:33:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:33:58.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:58 smithi105 bash[27669]: cluster 2024-02-18T01:33:57.956280+0000 mgr.y (mgr.24475) 656 : cluster [DBG] pgmap v1043: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 459 B/s wr, 1 op/s 2024-02-18T01:33:58.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:58 smithi105 bash[19734]: cluster 2024-02-18T01:33:57.956280+0000 mgr.y (mgr.24475) 656 : cluster [DBG] pgmap v1043: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 459 B/s wr, 1 op/s 2024-02-18T01:33:58.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:58 smithi155 bash[20805]: cluster 2024-02-18T01:33:57.956280+0000 mgr.y (mgr.24475) 656 : cluster [DBG] pgmap v1043: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 459 B/s wr, 1 op/s 2024-02-18T01:33:59.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:33:59 smithi105 bash[27669]: audit 2024-02-18T01:33:58.132059+0000 mgr.y (mgr.24475) 657 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:59.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:33:59 smithi105 bash[19734]: audit 2024-02-18T01:33:58.132059+0000 mgr.y (mgr.24475) 657 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:33:59.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:33:59 smithi155 bash[20805]: audit 2024-02-18T01:33:58.132059+0000 mgr.y (mgr.24475) 657 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:00.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:00 smithi105 bash[27669]: cluster 2024-02-18T01:33:59.957095+0000 mgr.y (mgr.24475) 658 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-18T01:34:00.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:00 smithi105 bash[19734]: cluster 2024-02-18T01:33:59.957095+0000 mgr.y (mgr.24475) 658 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-18T01:34:00.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:00 smithi155 bash[20805]: cluster 2024-02-18T01:33:59.957095+0000 mgr.y (mgr.24475) 658 : cluster [DBG] pgmap v1044: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 341 B/s wr, 1 op/s 2024-02-18T01:34:01.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:01 smithi105 bash[27669]: audit 2024-02-18T01:34:01.620199+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:01.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:01 smithi105 bash[19734]: audit 2024-02-18T01:34:01.620199+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:01.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:01.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:01 smithi155 bash[20805]: audit 2024-02-18T01:34:01.620199+0000 mon.a (mon.0) 2890 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:02.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[27669]: audit 2024-02-18T01:34:01.680338+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:02.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[27669]: cluster 2024-02-18T01:34:01.680472+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T01:34:02.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[27669]: audit 2024-02-18T01:34:01.682177+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[27669]: cluster 2024-02-18T01:34:01.957900+0000 mgr.y (mgr.24475) 659 : cluster [DBG] pgmap v1046: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[27669]: cluster 2024-02-18T01:34:02.673725+0000 mon.a (mon.0) 2894 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[19734]: audit 2024-02-18T01:34:01.680338+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[19734]: cluster 2024-02-18T01:34:01.680472+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[19734]: audit 2024-02-18T01:34:01.682177+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[19734]: cluster 2024-02-18T01:34:01.957900+0000 mgr.y (mgr.24475) 659 : cluster [DBG] pgmap v1046: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-02-18T01:34:02.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:02 smithi105 bash[19734]: cluster 2024-02-18T01:34:02.673725+0000 mon.a (mon.0) 2894 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:02 smithi155 bash[20805]: audit 2024-02-18T01:34:01.680338+0000 mon.a (mon.0) 2891 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:02 smithi155 bash[20805]: cluster 2024-02-18T01:34:01.680472+0000 mon.a (mon.0) 2892 : cluster [DBG] osdmap e611: 8 total, 8 up, 8 in 2024-02-18T01:34:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:02 smithi155 bash[20805]: audit 2024-02-18T01:34:01.682177+0000 mon.a (mon.0) 2893 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]: dispatch 2024-02-18T01:34:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:02 smithi155 bash[20805]: cluster 2024-02-18T01:34:01.957900+0000 mgr.y (mgr.24475) 659 : cluster [DBG] pgmap v1046: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 952 B/s rd, 317 B/s wr, 1 op/s 2024-02-18T01:34:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:02 smithi155 bash[20805]: cluster 2024-02-18T01:34:02.673725+0000 mon.a (mon.0) 2894 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:03.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:03 smithi105 bash[27669]: audit 2024-02-18T01:34:02.685813+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:34:03.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:03 smithi105 bash[27669]: cluster 2024-02-18T01:34:02.685922+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T01:34:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:03 smithi105 bash[19734]: audit 2024-02-18T01:34:02.685813+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:34:03.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:03 smithi105 bash[19734]: cluster 2024-02-18T01:34:02.685922+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T01:34:03.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:03 smithi155 bash[20805]: audit 2024-02-18T01:34:02.685813+0000 mon.a (mon.0) 2895 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-118"}]': finished 2024-02-18T01:34:03.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:03 smithi155 bash[20805]: cluster 2024-02-18T01:34:02.685922+0000 mon.a (mon.0) 2896 : cluster [DBG] osdmap e612: 8 total, 8 up, 8 in 2024-02-18T01:34:04.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:04 smithi105 bash[19734]: cluster 2024-02-18T01:34:03.709215+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T01:34:04.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:04 smithi105 bash[19734]: cluster 2024-02-18T01:34:03.958605+0000 mgr.y (mgr.24475) 660 : cluster [DBG] pgmap v1049: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:34:04.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:04 smithi105 bash[27669]: cluster 2024-02-18T01:34:03.709215+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T01:34:04.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:04 smithi105 bash[27669]: cluster 2024-02-18T01:34:03.958605+0000 mgr.y (mgr.24475) 660 : cluster [DBG] pgmap v1049: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:34:04.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:04 smithi155 bash[20805]: cluster 2024-02-18T01:34:03.709215+0000 mon.a (mon.0) 2897 : cluster [DBG] osdmap e613: 8 total, 8 up, 8 in 2024-02-18T01:34:04.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:04 smithi155 bash[20805]: cluster 2024-02-18T01:34:03.958605+0000 mgr.y (mgr.24475) 660 : cluster [DBG] pgmap v1049: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 0 op/s 2024-02-18T01:34:05.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:05 smithi105 bash[19734]: cluster 2024-02-18T01:34:04.723253+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T01:34:05.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:05 smithi105 bash[19734]: audit 2024-02-18T01:34:04.725355+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:05.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:05 smithi105 bash[27669]: cluster 2024-02-18T01:34:04.723253+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T01:34:05.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:05 smithi105 bash[27669]: audit 2024-02-18T01:34:04.725355+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:05.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:05 smithi155 bash[20805]: cluster 2024-02-18T01:34:04.723253+0000 mon.a (mon.0) 2898 : cluster [DBG] osdmap e614: 8 total, 8 up, 8 in 2024-02-18T01:34:05.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:05 smithi155 bash[20805]: audit 2024-02-18T01:34:04.725355+0000 mon.a (mon.0) 2899 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:06] "GET /metrics HTTP/1.1" 200 190676 "" "Prometheus/2.33.4" 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: audit 2024-02-18T01:34:05.726097+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: cluster 2024-02-18T01:34:05.726514+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: audit 2024-02-18T01:34:05.729837+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: cluster 2024-02-18T01:34:05.959425+0000 mgr.y (mgr.24475) 661 : cluster [DBG] pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: audit 2024-02-18T01:34:06.726421+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: cluster 2024-02-18T01:34:06.726503+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[19734]: audit 2024-02-18T01:34:06.727758+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: audit 2024-02-18T01:34:05.726097+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:06.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: cluster 2024-02-18T01:34:05.726514+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T01:34:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: audit 2024-02-18T01:34:05.729837+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: cluster 2024-02-18T01:34:05.959425+0000 mgr.y (mgr.24475) 661 : cluster [DBG] pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:34:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: audit 2024-02-18T01:34:06.726421+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: cluster 2024-02-18T01:34:06.726503+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T01:34:06.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:06 smithi105 bash[27669]: audit 2024-02-18T01:34:06.727758+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: audit 2024-02-18T01:34:05.726097+0000 mon.a (mon.0) 2900 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: cluster 2024-02-18T01:34:05.726514+0000 mon.a (mon.0) 2901 : cluster [DBG] osdmap e615: 8 total, 8 up, 8 in 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: audit 2024-02-18T01:34:05.729837+0000 mon.a (mon.0) 2902 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: cluster 2024-02-18T01:34:05.959425+0000 mgr.y (mgr.24475) 661 : cluster [DBG] pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: audit 2024-02-18T01:34:06.726421+0000 mon.a (mon.0) 2903 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: cluster 2024-02-18T01:34:06.726503+0000 mon.a (mon.0) 2904 : cluster [DBG] osdmap e616: 8 total, 8 up, 8 in 2024-02-18T01:34:06.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:06 smithi155 bash[20805]: audit 2024-02-18T01:34:06.727758+0000 mon.a (mon.0) 2905 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:08.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[19734]: audit 2024-02-18T01:34:07.730425+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:08.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[19734]: cluster 2024-02-18T01:34:07.731013+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[19734]: audit 2024-02-18T01:34:07.732095+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]: dispatch 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[19734]: cluster 2024-02-18T01:34:07.960297+0000 mgr.y (mgr.24475) 662 : cluster [DBG] pgmap v1055: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[27669]: audit 2024-02-18T01:34:07.730425+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[27669]: cluster 2024-02-18T01:34:07.731013+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[27669]: audit 2024-02-18T01:34:07.732095+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]: dispatch 2024-02-18T01:34:08.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:08 smithi105 bash[27669]: cluster 2024-02-18T01:34:07.960297+0000 mgr.y (mgr.24475) 662 : cluster [DBG] pgmap v1055: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:08 smithi155 bash[20805]: audit 2024-02-18T01:34:07.730425+0000 mon.a (mon.0) 2906 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:08 smithi155 bash[20805]: cluster 2024-02-18T01:34:07.731013+0000 mon.a (mon.0) 2907 : cluster [DBG] osdmap e617: 8 total, 8 up, 8 in 2024-02-18T01:34:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:08 smithi155 bash[20805]: audit 2024-02-18T01:34:07.732095+0000 mon.a (mon.0) 2908 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]: dispatch 2024-02-18T01:34:08.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:08 smithi155 bash[20805]: cluster 2024-02-18T01:34:07.960297+0000 mgr.y (mgr.24475) 662 : cluster [DBG] pgmap v1055: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:09.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[19734]: audit 2024-02-18T01:34:08.143434+0000 mgr.y (mgr.24475) 663 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:09.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[19734]: cluster 2024-02-18T01:34:08.723415+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:09.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[19734]: audit 2024-02-18T01:34:08.731888+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]': finished 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[19734]: cluster 2024-02-18T01:34:08.731982+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[19734]: audit 2024-02-18T01:34:08.922137+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[27669]: audit 2024-02-18T01:34:08.143434+0000 mgr.y (mgr.24475) 663 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[27669]: cluster 2024-02-18T01:34:08.723415+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[27669]: audit 2024-02-18T01:34:08.731888+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]': finished 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[27669]: cluster 2024-02-18T01:34:08.731982+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T01:34:09.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:09 smithi105 bash[27669]: audit 2024-02-18T01:34:08.922137+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:09 smithi155 bash[20805]: audit 2024-02-18T01:34:08.143434+0000 mgr.y (mgr.24475) 663 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:09 smithi155 bash[20805]: cluster 2024-02-18T01:34:08.723415+0000 mon.a (mon.0) 2909 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:09 smithi155 bash[20805]: audit 2024-02-18T01:34:08.731888+0000 mon.a (mon.0) 2910 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-120", "mode": "writeback"}]': finished 2024-02-18T01:34:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:09 smithi155 bash[20805]: cluster 2024-02-18T01:34:08.731982+0000 mon.a (mon.0) 2911 : cluster [DBG] osdmap e618: 8 total, 8 up, 8 in 2024-02-18T01:34:09.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:09 smithi155 bash[20805]: audit 2024-02-18T01:34:08.922137+0000 mon.a (mon.0) 2912 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[19734]: audit 2024-02-18T01:34:09.750231+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[19734]: cluster 2024-02-18T01:34:09.750508+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T01:34:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[19734]: audit 2024-02-18T01:34:09.751439+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:11.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[19734]: cluster 2024-02-18T01:34:09.961115+0000 mgr.y (mgr.24475) 664 : cluster [DBG] pgmap v1058: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[27669]: audit 2024-02-18T01:34:09.750231+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[27669]: cluster 2024-02-18T01:34:09.750508+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T01:34:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[27669]: audit 2024-02-18T01:34:09.751439+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:11.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:10 smithi105 bash[27669]: cluster 2024-02-18T01:34:09.961115+0000 mgr.y (mgr.24475) 664 : cluster [DBG] pgmap v1058: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:10 smithi155 bash[20805]: audit 2024-02-18T01:34:09.750231+0000 mon.a (mon.0) 2913 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:10 smithi155 bash[20805]: cluster 2024-02-18T01:34:09.750508+0000 mon.a (mon.0) 2914 : cluster [DBG] osdmap e619: 8 total, 8 up, 8 in 2024-02-18T01:34:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:10 smithi155 bash[20805]: audit 2024-02-18T01:34:09.751439+0000 mon.a (mon.0) 2915 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]: dispatch 2024-02-18T01:34:11.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:10 smithi155 bash[20805]: cluster 2024-02-18T01:34:09.961115+0000 mgr.y (mgr.24475) 664 : cluster [DBG] pgmap v1058: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:12.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[19734]: cluster 2024-02-18T01:34:10.747416+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:12.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[19734]: audit 2024-02-18T01:34:10.763483+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:12.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[19734]: cluster 2024-02-18T01:34:10.763607+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T01:34:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[27669]: cluster 2024-02-18T01:34:10.747416+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[27669]: audit 2024-02-18T01:34:10.763483+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:12.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:11 smithi105 bash[27669]: cluster 2024-02-18T01:34:10.763607+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T01:34:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:11 smithi155 bash[20805]: cluster 2024-02-18T01:34:10.747416+0000 mon.a (mon.0) 2916 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:11 smithi155 bash[20805]: audit 2024-02-18T01:34:10.763483+0000 mon.a (mon.0) 2917 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-120"}]': finished 2024-02-18T01:34:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:11 smithi155 bash[20805]: cluster 2024-02-18T01:34:10.763607+0000 mon.a (mon.0) 2918 : cluster [DBG] osdmap e620: 8 total, 8 up, 8 in 2024-02-18T01:34:13.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:12 smithi105 bash[19734]: cluster 2024-02-18T01:34:11.773603+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T01:34:13.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:12 smithi105 bash[19734]: cluster 2024-02-18T01:34:11.961889+0000 mgr.y (mgr.24475) 665 : cluster [DBG] pgmap v1061: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:13.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:12 smithi105 bash[27669]: cluster 2024-02-18T01:34:11.773603+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T01:34:13.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:12 smithi105 bash[27669]: cluster 2024-02-18T01:34:11.961889+0000 mgr.y (mgr.24475) 665 : cluster [DBG] pgmap v1061: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:12 smithi155 bash[20805]: cluster 2024-02-18T01:34:11.773603+0000 mon.a (mon.0) 2919 : cluster [DBG] osdmap e621: 8 total, 8 up, 8 in 2024-02-18T01:34:13.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:12 smithi155 bash[20805]: cluster 2024-02-18T01:34:11.961889+0000 mgr.y (mgr.24475) 665 : cluster [DBG] pgmap v1061: 172 pgs: 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:14.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:13 smithi105 bash[19734]: cluster 2024-02-18T01:34:12.788788+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T01:34:14.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:13 smithi105 bash[19734]: audit 2024-02-18T01:34:12.790307+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:14.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:13 smithi105 bash[27669]: cluster 2024-02-18T01:34:12.788788+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T01:34:14.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:13 smithi105 bash[27669]: audit 2024-02-18T01:34:12.790307+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:14.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:13 smithi155 bash[20805]: cluster 2024-02-18T01:34:12.788788+0000 mon.a (mon.0) 2920 : cluster [DBG] osdmap e622: 8 total, 8 up, 8 in 2024-02-18T01:34:14.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:13 smithi155 bash[20805]: audit 2024-02-18T01:34:12.790307+0000 mon.a (mon.0) 2921 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[19734]: audit 2024-02-18T01:34:13.796104+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:15.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[19734]: cluster 2024-02-18T01:34:13.796317+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[19734]: audit 2024-02-18T01:34:13.800463+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[19734]: cluster 2024-02-18T01:34:13.962722+0000 mgr.y (mgr.24475) 666 : cluster [DBG] pgmap v1064: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[27669]: audit 2024-02-18T01:34:13.796104+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[27669]: cluster 2024-02-18T01:34:13.796317+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[27669]: audit 2024-02-18T01:34:13.800463+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:15.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:14 smithi105 bash[27669]: cluster 2024-02-18T01:34:13.962722+0000 mgr.y (mgr.24475) 666 : cluster [DBG] pgmap v1064: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:14 smithi155 bash[20805]: audit 2024-02-18T01:34:13.796104+0000 mon.a (mon.0) 2922 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:14 smithi155 bash[20805]: cluster 2024-02-18T01:34:13.796317+0000 mon.a (mon.0) 2923 : cluster [DBG] osdmap e623: 8 total, 8 up, 8 in 2024-02-18T01:34:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:14 smithi155 bash[20805]: audit 2024-02-18T01:34:13.800463+0000 mon.a (mon.0) 2924 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:15.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:14 smithi155 bash[20805]: cluster 2024-02-18T01:34:13.962722+0000 mgr.y (mgr.24475) 666 : cluster [DBG] pgmap v1064: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:16.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: audit 2024-02-18T01:34:14.801608+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: cluster 2024-02-18T01:34:14.802156+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: audit 2024-02-18T01:34:14.803821+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: audit 2024-02-18T01:34:15.803961+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: cluster 2024-02-18T01:34:15.804062+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[19734]: audit 2024-02-18T01:34:15.805372+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]: dispatch 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: audit 2024-02-18T01:34:14.801608+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: cluster 2024-02-18T01:34:14.802156+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T01:34:16.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: audit 2024-02-18T01:34:14.803821+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:16.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: audit 2024-02-18T01:34:15.803961+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:16.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: cluster 2024-02-18T01:34:15.804062+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T01:34:16.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:15 smithi105 bash[27669]: audit 2024-02-18T01:34:15.805372+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]: dispatch 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: audit 2024-02-18T01:34:14.801608+0000 mon.a (mon.0) 2925 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: cluster 2024-02-18T01:34:14.802156+0000 mon.a (mon.0) 2926 : cluster [DBG] osdmap e624: 8 total, 8 up, 8 in 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: audit 2024-02-18T01:34:14.803821+0000 mon.a (mon.0) 2927 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: audit 2024-02-18T01:34:15.803961+0000 mon.a (mon.0) 2928 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: cluster 2024-02-18T01:34:15.804062+0000 mon.a (mon.0) 2929 : cluster [DBG] osdmap e625: 8 total, 8 up, 8 in 2024-02-18T01:34:16.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:15 smithi155 bash[20805]: audit 2024-02-18T01:34:15.805372+0000 mon.a (mon.0) 2930 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]: dispatch 2024-02-18T01:34:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:16] "GET /metrics HTTP/1.1" 200 190676 "" "Prometheus/2.33.4" 2024-02-18T01:34:16.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:16 smithi105 bash[27669]: cluster 2024-02-18T01:34:15.963580+0000 mgr.y (mgr.24475) 667 : cluster [DBG] pgmap v1067: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:16.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:16 smithi105 bash[27669]: cluster 2024-02-18T01:34:16.798070+0000 mon.a (mon.0) 2931 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:16 smithi105 bash[19734]: cluster 2024-02-18T01:34:15.963580+0000 mgr.y (mgr.24475) 667 : cluster [DBG] pgmap v1067: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:16.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:16 smithi105 bash[19734]: cluster 2024-02-18T01:34:16.798070+0000 mon.a (mon.0) 2931 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:16 smithi155 bash[20805]: cluster 2024-02-18T01:34:15.963580+0000 mgr.y (mgr.24475) 667 : cluster [DBG] pgmap v1067: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:17.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:16 smithi155 bash[20805]: cluster 2024-02-18T01:34:16.798070+0000 mon.a (mon.0) 2931 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:18.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:17 smithi155 bash[20805]: audit 2024-02-18T01:34:16.808194+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]': finished 2024-02-18T01:34:18.156 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:17 smithi155 bash[20805]: cluster 2024-02-18T01:34:16.808369+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T01:34:18.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:17 smithi105 bash[27669]: audit 2024-02-18T01:34:16.808194+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]': finished 2024-02-18T01:34:18.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:17 smithi105 bash[27669]: cluster 2024-02-18T01:34:16.808369+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T01:34:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:17 smithi105 bash[19734]: audit 2024-02-18T01:34:16.808194+0000 mon.a (mon.0) 2932 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-122", "mode": "writeback"}]': finished 2024-02-18T01:34:18.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:17 smithi105 bash[19734]: cluster 2024-02-18T01:34:16.808369+0000 mon.a (mon.0) 2933 : cluster [DBG] osdmap e626: 8 total, 8 up, 8 in 2024-02-18T01:34:18.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:19.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:18 smithi105 bash[27669]: cluster 2024-02-18T01:34:17.824960+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T01:34:19.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:18 smithi105 bash[27669]: cluster 2024-02-18T01:34:17.964271+0000 mgr.y (mgr.24475) 668 : cluster [DBG] pgmap v1070: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:18 smithi105 bash[19734]: cluster 2024-02-18T01:34:17.824960+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T01:34:19.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:18 smithi105 bash[19734]: cluster 2024-02-18T01:34:17.964271+0000 mgr.y (mgr.24475) 668 : cluster [DBG] pgmap v1070: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:18 smithi155 bash[20805]: cluster 2024-02-18T01:34:17.824960+0000 mon.a (mon.0) 2934 : cluster [DBG] osdmap e627: 8 total, 8 up, 8 in 2024-02-18T01:34:19.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:18 smithi155 bash[20805]: cluster 2024-02-18T01:34:17.964271+0000 mgr.y (mgr.24475) 668 : cluster [DBG] pgmap v1070: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[19734]: audit 2024-02-18T01:34:18.154867+0000 mgr.y (mgr.24475) 669 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[19734]: cluster 2024-02-18T01:34:18.835292+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T01:34:20.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[19734]: audit 2024-02-18T01:34:18.882760+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[27669]: audit 2024-02-18T01:34:18.154867+0000 mgr.y (mgr.24475) 669 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[27669]: cluster 2024-02-18T01:34:18.835292+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T01:34:20.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:19 smithi105 bash[27669]: audit 2024-02-18T01:34:18.882760+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:20.242 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:19 smithi155 bash[20805]: audit 2024-02-18T01:34:18.154867+0000 mgr.y (mgr.24475) 669 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:19 smithi155 bash[20805]: cluster 2024-02-18T01:34:18.835292+0000 mon.a (mon.0) 2935 : cluster [DBG] osdmap e628: 8 total, 8 up, 8 in 2024-02-18T01:34:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:19 smithi155 bash[20805]: audit 2024-02-18T01:34:18.882760+0000 mon.a (mon.0) 2936 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[19734]: audit 2024-02-18T01:34:19.859593+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[19734]: cluster 2024-02-18T01:34:19.859687+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T01:34:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[19734]: audit 2024-02-18T01:34:19.870658+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:21.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[19734]: cluster 2024-02-18T01:34:19.965137+0000 mgr.y (mgr.24475) 670 : cluster [DBG] pgmap v1073: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[27669]: audit 2024-02-18T01:34:19.859593+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[27669]: cluster 2024-02-18T01:34:19.859687+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T01:34:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[27669]: audit 2024-02-18T01:34:19.870658+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:21.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:20 smithi105 bash[27669]: cluster 2024-02-18T01:34:19.965137+0000 mgr.y (mgr.24475) 670 : cluster [DBG] pgmap v1073: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:20 smithi155 bash[20805]: audit 2024-02-18T01:34:19.859593+0000 mon.a (mon.0) 2937 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:20 smithi155 bash[20805]: cluster 2024-02-18T01:34:19.859687+0000 mon.a (mon.0) 2938 : cluster [DBG] osdmap e629: 8 total, 8 up, 8 in 2024-02-18T01:34:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:20 smithi155 bash[20805]: audit 2024-02-18T01:34:19.870658+0000 mon.a (mon.0) 2939 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:21.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:20 smithi155 bash[20805]: cluster 2024-02-18T01:34:19.965137+0000 mgr.y (mgr.24475) 670 : cluster [DBG] pgmap v1073: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:34:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[19734]: audit 2024-02-18T01:34:20.859316+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[19734]: cluster 2024-02-18T01:34:20.859399+0000 mon.a (mon.0) 2941 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T01:34:22.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[19734]: audit 2024-02-18T01:34:21.520729+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[27669]: audit 2024-02-18T01:34:20.859316+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[27669]: cluster 2024-02-18T01:34:20.859399+0000 mon.a (mon.0) 2941 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T01:34:22.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:21 smithi105 bash[27669]: audit 2024-02-18T01:34:21.520729+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:21 smithi155 bash[20805]: audit 2024-02-18T01:34:20.859316+0000 mon.a (mon.0) 2940 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:21 smithi155 bash[20805]: cluster 2024-02-18T01:34:20.859399+0000 mon.a (mon.0) 2941 : cluster [DBG] osdmap e630: 8 total, 8 up, 8 in 2024-02-18T01:34:22.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:21 smithi155 bash[20805]: audit 2024-02-18T01:34:21.520729+0000 mon.a (mon.0) 2942 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[19734]: cluster 2024-02-18T01:34:21.873943+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T01:34:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[19734]: audit 2024-02-18T01:34:21.903453+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:23.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[19734]: cluster 2024-02-18T01:34:21.965984+0000 mgr.y (mgr.24475) 671 : cluster [DBG] pgmap v1076: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:23.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[27669]: cluster 2024-02-18T01:34:21.873943+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T01:34:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[27669]: audit 2024-02-18T01:34:21.903453+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:23.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:22 smithi105 bash[27669]: cluster 2024-02-18T01:34:21.965984+0000 mgr.y (mgr.24475) 671 : cluster [DBG] pgmap v1076: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:22 smithi155 bash[20805]: cluster 2024-02-18T01:34:21.873943+0000 mon.a (mon.0) 2943 : cluster [DBG] osdmap e631: 8 total, 8 up, 8 in 2024-02-18T01:34:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:22 smithi155 bash[20805]: audit 2024-02-18T01:34:21.903453+0000 mon.a (mon.0) 2944 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:23.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:22 smithi155 bash[20805]: cluster 2024-02-18T01:34:21.965984+0000 mgr.y (mgr.24475) 671 : cluster [DBG] pgmap v1076: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:24.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[27669]: audit 2024-02-18T01:34:22.895541+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:24.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[27669]: cluster 2024-02-18T01:34:22.895662+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T01:34:24.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[27669]: audit 2024-02-18T01:34:22.896909+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[19734]: audit 2024-02-18T01:34:22.895541+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[19734]: cluster 2024-02-18T01:34:22.895662+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T01:34:24.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:23 smithi105 bash[19734]: audit 2024-02-18T01:34:22.896909+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:23 smithi155 bash[20805]: audit 2024-02-18T01:34:22.895541+0000 mon.a (mon.0) 2945 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:23 smithi155 bash[20805]: cluster 2024-02-18T01:34:22.895662+0000 mon.a (mon.0) 2946 : cluster [DBG] osdmap e632: 8 total, 8 up, 8 in 2024-02-18T01:34:24.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:23 smithi155 bash[20805]: audit 2024-02-18T01:34:22.896909+0000 mon.a (mon.0) 2947 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]: dispatch 2024-02-18T01:34:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[27669]: cluster 2024-02-18T01:34:23.890670+0000 mon.a (mon.0) 2948 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[27669]: audit 2024-02-18T01:34:23.903410+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[27669]: cluster 2024-02-18T01:34:23.903483+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T01:34:25.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[27669]: cluster 2024-02-18T01:34:23.966782+0000 mgr.y (mgr.24475) 672 : cluster [DBG] pgmap v1079: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:34:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[19734]: cluster 2024-02-18T01:34:23.890670+0000 mon.a (mon.0) 2948 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[19734]: audit 2024-02-18T01:34:23.903410+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[19734]: cluster 2024-02-18T01:34:23.903483+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T01:34:25.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:24 smithi105 bash[19734]: cluster 2024-02-18T01:34:23.966782+0000 mgr.y (mgr.24475) 672 : cluster [DBG] pgmap v1079: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:34:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:24 smithi155 bash[20805]: cluster 2024-02-18T01:34:23.890670+0000 mon.a (mon.0) 2948 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:24 smithi155 bash[20805]: audit 2024-02-18T01:34:23.903410+0000 mon.a (mon.0) 2949 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-122"}]': finished 2024-02-18T01:34:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:24 smithi155 bash[20805]: cluster 2024-02-18T01:34:23.903483+0000 mon.a (mon.0) 2950 : cluster [DBG] osdmap e633: 8 total, 8 up, 8 in 2024-02-18T01:34:25.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:24 smithi155 bash[20805]: cluster 2024-02-18T01:34:23.966782+0000 mgr.y (mgr.24475) 672 : cluster [DBG] pgmap v1079: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-02-18T01:34:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:25 smithi105 bash[19734]: cluster 2024-02-18T01:34:24.901877+0000 mon.a (mon.0) 2951 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:26.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:25 smithi105 bash[19734]: cluster 2024-02-18T01:34:24.919221+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T01:34:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:25 smithi105 bash[27669]: cluster 2024-02-18T01:34:24.901877+0000 mon.a (mon.0) 2951 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:26.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:25 smithi105 bash[27669]: cluster 2024-02-18T01:34:24.919221+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T01:34:26.242 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:25 smithi155 bash[20805]: cluster 2024-02-18T01:34:24.901877+0000 mon.a (mon.0) 2951 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:26.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:25 smithi155 bash[20805]: cluster 2024-02-18T01:34:24.919221+0000 mon.a (mon.0) 2952 : cluster [DBG] osdmap e634: 8 total, 8 up, 8 in 2024-02-18T01:34:26.930 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:26] "GET /metrics HTTP/1.1" 200 190834 "" "Prometheus/2.33.4" 2024-02-18T01:34:26.930 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[19734]: cluster 2024-02-18T01:34:25.921411+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T01:34:26.930 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[19734]: audit 2024-02-18T01:34:25.924800+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:26.930 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[19734]: cluster 2024-02-18T01:34:25.967406+0000 mgr.y (mgr.24475) 673 : cluster [DBG] pgmap v1082: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:34:27.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[27669]: cluster 2024-02-18T01:34:25.921411+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T01:34:27.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[27669]: audit 2024-02-18T01:34:25.924800+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:27.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:26 smithi105 bash[27669]: cluster 2024-02-18T01:34:25.967406+0000 mgr.y (mgr.24475) 673 : cluster [DBG] pgmap v1082: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:34:27.242 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:26 smithi155 bash[20805]: cluster 2024-02-18T01:34:25.921411+0000 mon.a (mon.0) 2953 : cluster [DBG] osdmap e635: 8 total, 8 up, 8 in 2024-02-18T01:34:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:26 smithi155 bash[20805]: audit 2024-02-18T01:34:25.924800+0000 mon.a (mon.0) 2954 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:27.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:26 smithi155 bash[20805]: cluster 2024-02-18T01:34:25.967406+0000 mgr.y (mgr.24475) 673 : cluster [DBG] pgmap v1082: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:34:28.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:26.934532+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: cluster 2024-02-18T01:34:26.934752+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:26.936874+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:27.121051+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:27.435447+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:27.436845+0000 mon.a (mon.0) 2960 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[19734]: audit 2024-02-18T01:34:27.444311+0000 mon.a (mon.0) 2961 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:26.934532+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: cluster 2024-02-18T01:34:26.934752+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T01:34:28.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:26.936874+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:27.121051+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:34:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:27.435447+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:34:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:27.436845+0000 mon.a (mon.0) 2960 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:34:28.241 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:27 smithi105 bash[27669]: audit 2024-02-18T01:34:27.444311+0000 mon.a (mon.0) 2961 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:28.243 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:26.934532+0000 mon.a (mon.0) 2955 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: cluster 2024-02-18T01:34:26.934752+0000 mon.a (mon.0) 2956 : cluster [DBG] osdmap e636: 8 total, 8 up, 8 in 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:26.936874+0000 mon.a (mon.0) 2957 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:27.121051+0000 mon.a (mon.0) 2958 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:27.435447+0000 mon.a (mon.0) 2959 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:27.436845+0000 mon.a (mon.0) 2960 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:34:28.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:27 smithi155 bash[20805]: audit 2024-02-18T01:34:27.444311+0000 mon.a (mon.0) 2961 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:29.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[19734]: audit 2024-02-18T01:34:27.938023+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:29.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[19734]: cluster 2024-02-18T01:34:27.938240+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[19734]: audit 2024-02-18T01:34:27.939316+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[19734]: cluster 2024-02-18T01:34:27.968304+0000 mgr.y (mgr.24475) 674 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[19734]: audit 2024-02-18T01:34:28.941048+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[27669]: audit 2024-02-18T01:34:27.938023+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[27669]: cluster 2024-02-18T01:34:27.938240+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[27669]: audit 2024-02-18T01:34:27.939316+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[27669]: cluster 2024-02-18T01:34:27.968304+0000 mgr.y (mgr.24475) 674 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:34:29.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:28 smithi105 bash[27669]: audit 2024-02-18T01:34:28.941048+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[20805]: audit 2024-02-18T01:34:27.938023+0000 mon.a (mon.0) 2962 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[20805]: cluster 2024-02-18T01:34:27.938240+0000 mon.a (mon.0) 2963 : cluster [DBG] osdmap e637: 8 total, 8 up, 8 in 2024-02-18T01:34:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[20805]: audit 2024-02-18T01:34:27.939316+0000 mon.a (mon.0) 2964 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[20805]: cluster 2024-02-18T01:34:27.968304+0000 mgr.y (mgr.24475) 674 : cluster [DBG] pgmap v1085: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-02-18T01:34:29.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:28 smithi155 bash[20805]: audit 2024-02-18T01:34:28.941048+0000 mon.a (mon.0) 2965 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:30.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: audit 2024-02-18T01:34:28.165772+0000 mgr.y (mgr.24475) 675 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:30.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: cluster 2024-02-18T01:34:28.941256+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T01:34:30.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: audit 2024-02-18T01:34:28.942820+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]: dispatch 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: cluster 2024-02-18T01:34:29.489260+0000 mon.a (mon.0) 2968 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: audit 2024-02-18T01:34:29.499964+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]': finished 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: cluster 2024-02-18T01:34:29.500161+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[27669]: audit 2024-02-18T01:34:29.548248+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: audit 2024-02-18T01:34:28.165772+0000 mgr.y (mgr.24475) 675 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: cluster 2024-02-18T01:34:28.941256+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: audit 2024-02-18T01:34:28.942820+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]: dispatch 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: cluster 2024-02-18T01:34:29.489260+0000 mon.a (mon.0) 2968 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:30.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: audit 2024-02-18T01:34:29.499964+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]': finished 2024-02-18T01:34:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: cluster 2024-02-18T01:34:29.500161+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T01:34:30.241 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:29 smithi105 bash[19734]: audit 2024-02-18T01:34:29.548248+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: audit 2024-02-18T01:34:28.165772+0000 mgr.y (mgr.24475) 675 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: cluster 2024-02-18T01:34:28.941256+0000 mon.a (mon.0) 2966 : cluster [DBG] osdmap e638: 8 total, 8 up, 8 in 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: audit 2024-02-18T01:34:28.942820+0000 mon.a (mon.0) 2967 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]: dispatch 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: cluster 2024-02-18T01:34:29.489260+0000 mon.a (mon.0) 2968 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: audit 2024-02-18T01:34:29.499964+0000 mon.a (mon.0) 2969 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-124", "mode": "writeback"}]': finished 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: cluster 2024-02-18T01:34:29.500161+0000 mon.a (mon.0) 2970 : cluster [DBG] osdmap e639: 8 total, 8 up, 8 in 2024-02-18T01:34:30.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:29 smithi155 bash[20805]: audit 2024-02-18T01:34:29.548248+0000 mon.a (mon.0) 2971 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:31.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[19734]: cluster 2024-02-18T01:34:29.969084+0000 mgr.y (mgr.24475) 676 : cluster [DBG] pgmap v1088: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:31.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[19734]: audit 2024-02-18T01:34:30.500899+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:31.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[19734]: cluster 2024-02-18T01:34:30.501062+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T01:34:31.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[19734]: audit 2024-02-18T01:34:30.503215+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[27669]: cluster 2024-02-18T01:34:29.969084+0000 mgr.y (mgr.24475) 676 : cluster [DBG] pgmap v1088: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[27669]: audit 2024-02-18T01:34:30.500899+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[27669]: cluster 2024-02-18T01:34:30.501062+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T01:34:31.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:30 smithi105 bash[27669]: audit 2024-02-18T01:34:30.503215+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:31.242 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:30 smithi155 bash[20805]: cluster 2024-02-18T01:34:29.969084+0000 mgr.y (mgr.24475) 676 : cluster [DBG] pgmap v1088: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:30 smithi155 bash[20805]: audit 2024-02-18T01:34:30.500899+0000 mon.a (mon.0) 2972 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:30 smithi155 bash[20805]: cluster 2024-02-18T01:34:30.501062+0000 mon.a (mon.0) 2973 : cluster [DBG] osdmap e640: 8 total, 8 up, 8 in 2024-02-18T01:34:31.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:30 smithi155 bash[20805]: audit 2024-02-18T01:34:30.503215+0000 mon.a (mon.0) 2974 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]: dispatch 2024-02-18T01:34:32.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[19734]: cluster 2024-02-18T01:34:31.494662+0000 mon.a (mon.0) 2975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:32.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[19734]: audit 2024-02-18T01:34:31.504423+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:32.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[19734]: cluster 2024-02-18T01:34:31.504604+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T01:34:32.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[27669]: cluster 2024-02-18T01:34:31.494662+0000 mon.a (mon.0) 2975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[27669]: audit 2024-02-18T01:34:31.504423+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:32.240 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:31 smithi105 bash[27669]: cluster 2024-02-18T01:34:31.504604+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T01:34:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:31 smithi155 bash[20805]: cluster 2024-02-18T01:34:31.494662+0000 mon.a (mon.0) 2975 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:31 smithi155 bash[20805]: audit 2024-02-18T01:34:31.504423+0000 mon.a (mon.0) 2976 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-124"}]': finished 2024-02-18T01:34:32.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:31 smithi155 bash[20805]: cluster 2024-02-18T01:34:31.504604+0000 mon.a (mon.0) 2977 : cluster [DBG] osdmap e641: 8 total, 8 up, 8 in 2024-02-18T01:34:33.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:32 smithi105 bash[27669]: cluster 2024-02-18T01:34:31.969696+0000 mgr.y (mgr.24475) 677 : cluster [DBG] pgmap v1091: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:33.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:32 smithi105 bash[27669]: cluster 2024-02-18T01:34:32.506474+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T01:34:33.239 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:32 smithi105 bash[19734]: cluster 2024-02-18T01:34:31.969696+0000 mgr.y (mgr.24475) 677 : cluster [DBG] pgmap v1091: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:32 smithi105 bash[19734]: cluster 2024-02-18T01:34:32.506474+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T01:34:33.242 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:32 smithi155 bash[20805]: cluster 2024-02-18T01:34:31.969696+0000 mgr.y (mgr.24475) 677 : cluster [DBG] pgmap v1091: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:32 smithi155 bash[20805]: cluster 2024-02-18T01:34:32.506474+0000 mon.a (mon.0) 2978 : cluster [DBG] osdmap e642: 8 total, 8 up, 8 in 2024-02-18T01:34:34.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[27669]: cluster 2024-02-18T01:34:33.510610+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T01:34:34.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[27669]: audit 2024-02-18T01:34:33.514322+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:34.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[27669]: cluster 2024-02-18T01:34:33.970442+0000 mgr.y (mgr.24475) 678 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:34.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[27669]: cluster 2024-02-18T01:34:34.490935+0000 mon.a (mon.0) 2981 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[19734]: cluster 2024-02-18T01:34:33.510610+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T01:34:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[19734]: audit 2024-02-18T01:34:33.514322+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[19734]: cluster 2024-02-18T01:34:33.970442+0000 mgr.y (mgr.24475) 678 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:34.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:34 smithi105 bash[19734]: cluster 2024-02-18T01:34:34.490935+0000 mon.a (mon.0) 2981 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:34 smithi155 bash[20805]: cluster 2024-02-18T01:34:33.510610+0000 mon.a (mon.0) 2979 : cluster [DBG] osdmap e643: 8 total, 8 up, 8 in 2024-02-18T01:34:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:34 smithi155 bash[20805]: audit 2024-02-18T01:34:33.514322+0000 mon.a (mon.0) 2980 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:34 smithi155 bash[20805]: cluster 2024-02-18T01:34:33.970442+0000 mgr.y (mgr.24475) 678 : cluster [DBG] pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:34.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:34 smithi155 bash[20805]: cluster 2024-02-18T01:34:34.490935+0000 mon.a (mon.0) 2981 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:34:35.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: audit 2024-02-18T01:34:34.510885+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:35.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: cluster 2024-02-18T01:34:34.511053+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T01:34:35.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: audit 2024-02-18T01:34:34.514825+0000 mon.a (mon.0) 2984 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: audit 2024-02-18T01:34:35.515109+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: cluster 2024-02-18T01:34:35.515317+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[19734]: audit 2024-02-18T01:34:35.516066+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: audit 2024-02-18T01:34:34.510885+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: cluster 2024-02-18T01:34:34.511053+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: audit 2024-02-18T01:34:34.514825+0000 mon.a (mon.0) 2984 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: audit 2024-02-18T01:34:35.515109+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: cluster 2024-02-18T01:34:35.515317+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T01:34:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:35 smithi105 bash[27669]: audit 2024-02-18T01:34:35.516066+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: audit 2024-02-18T01:34:34.510885+0000 mon.a (mon.0) 2982 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: cluster 2024-02-18T01:34:34.511053+0000 mon.a (mon.0) 2983 : cluster [DBG] osdmap e644: 8 total, 8 up, 8 in 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: audit 2024-02-18T01:34:34.514825+0000 mon.a (mon.0) 2984 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: audit 2024-02-18T01:34:35.515109+0000 mon.a (mon.0) 2985 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: cluster 2024-02-18T01:34:35.515317+0000 mon.a (mon.0) 2986 : cluster [DBG] osdmap e645: 8 total, 8 up, 8 in 2024-02-18T01:34:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:35 smithi155 bash[20805]: audit 2024-02-18T01:34:35.516066+0000 mon.a (mon.0) 2987 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:36] "GET /metrics HTTP/1.1" 200 190936 "" "Prometheus/2.33.4" 2024-02-18T01:34:36.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[19734]: cluster 2024-02-18T01:34:35.971285+0000 mgr.y (mgr.24475) 679 : cluster [DBG] pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[19734]: audit 2024-02-18T01:34:36.520093+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[19734]: cluster 2024-02-18T01:34:36.520274+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[19734]: audit 2024-02-18T01:34:36.521446+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]: dispatch 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[27669]: cluster 2024-02-18T01:34:35.971285+0000 mgr.y (mgr.24475) 679 : cluster [DBG] pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[27669]: audit 2024-02-18T01:34:36.520093+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[27669]: cluster 2024-02-18T01:34:36.520274+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T01:34:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:36 smithi105 bash[27669]: audit 2024-02-18T01:34:36.521446+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]: dispatch 2024-02-18T01:34:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:36 smithi155 bash[20805]: cluster 2024-02-18T01:34:35.971285+0000 mgr.y (mgr.24475) 679 : cluster [DBG] pgmap v1097: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-02-18T01:34:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:36 smithi155 bash[20805]: audit 2024-02-18T01:34:36.520093+0000 mon.a (mon.0) 2988 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:36 smithi155 bash[20805]: cluster 2024-02-18T01:34:36.520274+0000 mon.a (mon.0) 2989 : cluster [DBG] osdmap e646: 8 total, 8 up, 8 in 2024-02-18T01:34:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:36 smithi155 bash[20805]: audit 2024-02-18T01:34:36.521446+0000 mon.a (mon.0) 2990 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]: dispatch 2024-02-18T01:34:37.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[27669]: audit 2024-02-18T01:34:36.527567+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:37.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[27669]: cluster 2024-02-18T01:34:37.511751+0000 mon.a (mon.0) 2992 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:37.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[27669]: audit 2024-02-18T01:34:37.521205+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]': finished 2024-02-18T01:34:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[27669]: cluster 2024-02-18T01:34:37.521345+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T01:34:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[19734]: audit 2024-02-18T01:34:36.527567+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[19734]: cluster 2024-02-18T01:34:37.511751+0000 mon.a (mon.0) 2992 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[19734]: audit 2024-02-18T01:34:37.521205+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]': finished 2024-02-18T01:34:37.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:37 smithi105 bash[19734]: cluster 2024-02-18T01:34:37.521345+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T01:34:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:37 smithi155 bash[20805]: audit 2024-02-18T01:34:36.527567+0000 mon.a (mon.0) 2991 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:34:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:37 smithi155 bash[20805]: cluster 2024-02-18T01:34:37.511751+0000 mon.a (mon.0) 2992 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:34:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:37 smithi155 bash[20805]: audit 2024-02-18T01:34:37.521205+0000 mon.a (mon.0) 2993 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-126", "mode": "writeback"}]': finished 2024-02-18T01:34:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:37 smithi155 bash[20805]: cluster 2024-02-18T01:34:37.521345+0000 mon.a (mon.0) 2994 : cluster [DBG] osdmap e647: 8 total, 8 up, 8 in 2024-02-18T01:34:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:38 smithi105 bash[27669]: audit 2024-02-18T01:34:37.592929+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:38 smithi105 bash[27669]: cluster 2024-02-18T01:34:37.972132+0000 mgr.y (mgr.24475) 680 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:38.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:38 smithi105 bash[19734]: audit 2024-02-18T01:34:37.592929+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:38 smithi105 bash[19734]: cluster 2024-02-18T01:34:37.972132+0000 mgr.y (mgr.24475) 680 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:38 smithi155 bash[20805]: audit 2024-02-18T01:34:37.592929+0000 mon.a (mon.0) 2995 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:38.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:38 smithi155 bash[20805]: cluster 2024-02-18T01:34:37.972132+0000 mgr.y (mgr.24475) 680 : cluster [DBG] pgmap v1100: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[27669]: audit 2024-02-18T01:34:38.177092+0000 mgr.y (mgr.24475) 681 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[27669]: audit 2024-02-18T01:34:38.553174+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[27669]: cluster 2024-02-18T01:34:38.553285+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T01:34:39.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[27669]: audit 2024-02-18T01:34:38.554258+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[19734]: audit 2024-02-18T01:34:38.177092+0000 mgr.y (mgr.24475) 681 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[19734]: audit 2024-02-18T01:34:38.553174+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[19734]: cluster 2024-02-18T01:34:38.553285+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T01:34:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:39 smithi105 bash[19734]: audit 2024-02-18T01:34:38.554258+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:39 smithi155 bash[20805]: audit 2024-02-18T01:34:38.177092+0000 mgr.y (mgr.24475) 681 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:39 smithi155 bash[20805]: audit 2024-02-18T01:34:38.553174+0000 mon.a (mon.0) 2996 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:34:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:39 smithi155 bash[20805]: cluster 2024-02-18T01:34:38.553285+0000 mon.a (mon.0) 2997 : cluster [DBG] osdmap e648: 8 total, 8 up, 8 in 2024-02-18T01:34:39.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:39 smithi155 bash[20805]: audit 2024-02-18T01:34:38.554258+0000 mon.a (mon.0) 2998 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]: dispatch 2024-02-18T01:34:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[19734]: cluster 2024-02-18T01:34:39.547547+0000 mon.a (mon.0) 2999 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[19734]: audit 2024-02-18T01:34:39.558859+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[19734]: cluster 2024-02-18T01:34:39.559044+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T01:34:40.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[19734]: cluster 2024-02-18T01:34:39.972997+0000 mgr.y (mgr.24475) 682 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[27669]: cluster 2024-02-18T01:34:39.547547+0000 mon.a (mon.0) 2999 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[27669]: audit 2024-02-18T01:34:39.558859+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[27669]: cluster 2024-02-18T01:34:39.559044+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T01:34:40.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:40 smithi105 bash[27669]: cluster 2024-02-18T01:34:39.972997+0000 mgr.y (mgr.24475) 682 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:40.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:40 smithi155 bash[20805]: cluster 2024-02-18T01:34:39.547547+0000 mon.a (mon.0) 2999 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:34:40.995 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:40 smithi155 bash[20805]: audit 2024-02-18T01:34:39.558859+0000 mon.a (mon.0) 3000 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-126"}]': finished 2024-02-18T01:34:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:40 smithi155 bash[20805]: cluster 2024-02-18T01:34:39.559044+0000 mon.a (mon.0) 3001 : cluster [DBG] osdmap e649: 8 total, 8 up, 8 in 2024-02-18T01:34:40.996 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:40 smithi155 bash[20805]: cluster 2024-02-18T01:34:39.972997+0000 mgr.y (mgr.24475) 682 : cluster [DBG] pgmap v1103: 204 pgs: 204 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:41.876 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:41 smithi155 bash[20805]: cluster 2024-02-18T01:34:40.575782+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T01:34:41.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:41 smithi105 bash[19734]: cluster 2024-02-18T01:34:40.575782+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T01:34:41.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:41 smithi105 bash[27669]: cluster 2024-02-18T01:34:40.575782+0000 mon.a (mon.0) 3002 : cluster [DBG] osdmap e650: 8 total, 8 up, 8 in 2024-02-18T01:34:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[19734]: cluster 2024-02-18T01:34:41.591472+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T01:34:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[19734]: audit 2024-02-18T01:34:41.594942+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[19734]: cluster 2024-02-18T01:34:41.973865+0000 mgr.y (mgr.24475) 683 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[27669]: cluster 2024-02-18T01:34:41.591472+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T01:34:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[27669]: audit 2024-02-18T01:34:41.594942+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:42 smithi105 bash[27669]: cluster 2024-02-18T01:34:41.973865+0000 mgr.y (mgr.24475) 683 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:42 smithi155 bash[20805]: cluster 2024-02-18T01:34:41.591472+0000 mon.a (mon.0) 3003 : cluster [DBG] osdmap e651: 8 total, 8 up, 8 in 2024-02-18T01:34:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:42 smithi155 bash[20805]: audit 2024-02-18T01:34:41.594942+0000 mon.a (mon.0) 3004 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:42 smithi155 bash[20805]: cluster 2024-02-18T01:34:41.973865+0000 mgr.y (mgr.24475) 683 : cluster [DBG] pgmap v1106: 204 pgs: 32 unknown, 172 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:43.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: audit 2024-02-18T01:34:42.594753+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: cluster 2024-02-18T01:34:42.595211+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: audit 2024-02-18T01:34:42.600017+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: audit 2024-02-18T01:34:43.596434+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: cluster 2024-02-18T01:34:43.596627+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[27669]: audit 2024-02-18T01:34:43.598095+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: audit 2024-02-18T01:34:42.594753+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: cluster 2024-02-18T01:34:42.595211+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: audit 2024-02-18T01:34:42.600017+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: audit 2024-02-18T01:34:43.596434+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: cluster 2024-02-18T01:34:43.596627+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T01:34:43.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:43 smithi105 bash[19734]: audit 2024-02-18T01:34:43.598095+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: audit 2024-02-18T01:34:42.594753+0000 mon.a (mon.0) 3005 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: cluster 2024-02-18T01:34:42.595211+0000 mon.a (mon.0) 3006 : cluster [DBG] osdmap e652: 8 total, 8 up, 8 in 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: audit 2024-02-18T01:34:42.600017+0000 mon.a (mon.0) 3007 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: audit 2024-02-18T01:34:43.596434+0000 mon.a (mon.0) 3008 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: cluster 2024-02-18T01:34:43.596627+0000 mon.a (mon.0) 3009 : cluster [DBG] osdmap e653: 8 total, 8 up, 8 in 2024-02-18T01:34:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:43 smithi155 bash[20805]: audit 2024-02-18T01:34:43.598095+0000 mon.a (mon.0) 3010 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:34:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[19734]: cluster 2024-02-18T01:34:43.974741+0000 mgr.y (mgr.24475) 684 : cluster [DBG] pgmap v1109: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[19734]: audit 2024-02-18T01:34:44.603245+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:34:44.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[19734]: cluster 2024-02-18T01:34:44.603455+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T01:34:44.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[19734]: audit 2024-02-18T01:34:44.604439+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:34:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[27669]: cluster 2024-02-18T01:34:43.974741+0000 mgr.y (mgr.24475) 684 : cluster [DBG] pgmap v1109: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[27669]: audit 2024-02-18T01:34:44.603245+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:34:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[27669]: cluster 2024-02-18T01:34:44.603455+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T01:34:44.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:44 smithi105 bash[27669]: audit 2024-02-18T01:34:44.604439+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:34:44.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:44 smithi155 bash[20805]: cluster 2024-02-18T01:34:43.974741+0000 mgr.y (mgr.24475) 684 : cluster [DBG] pgmap v1109: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:44 smithi155 bash[20805]: audit 2024-02-18T01:34:44.603245+0000 mon.a (mon.0) 3011 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:34:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:44 smithi155 bash[20805]: cluster 2024-02-18T01:34:44.603455+0000 mon.a (mon.0) 3012 : cluster [DBG] osdmap e654: 8 total, 8 up, 8 in 2024-02-18T01:34:44.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:44 smithi155 bash[20805]: audit 2024-02-18T01:34:44.604439+0000 mon.a (mon.0) 3013 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:34:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:46] "GET /metrics HTTP/1.1" 200 190936 "" "Prometheus/2.33.4" 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[27669]: audit 2024-02-18T01:34:45.604435+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[27669]: cluster 2024-02-18T01:34:45.604567+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[27669]: audit 2024-02-18T01:34:45.605417+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[27669]: cluster 2024-02-18T01:34:45.975649+0000 mgr.y (mgr.24475) 685 : cluster [DBG] pgmap v1112: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[19734]: audit 2024-02-18T01:34:45.604435+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[19734]: cluster 2024-02-18T01:34:45.604567+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[19734]: audit 2024-02-18T01:34:45.605417+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:34:46.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:46 smithi105 bash[19734]: cluster 2024-02-18T01:34:45.975649+0000 mgr.y (mgr.24475) 685 : cluster [DBG] pgmap v1112: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:46 smithi155 bash[20805]: audit 2024-02-18T01:34:45.604435+0000 mon.a (mon.0) 3014 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:34:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:46 smithi155 bash[20805]: cluster 2024-02-18T01:34:45.604567+0000 mon.a (mon.0) 3015 : cluster [DBG] osdmap e655: 8 total, 8 up, 8 in 2024-02-18T01:34:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:46 smithi155 bash[20805]: audit 2024-02-18T01:34:45.605417+0000 mon.a (mon.0) 3016 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-02-18T01:34:46.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:46 smithi155 bash[20805]: cluster 2024-02-18T01:34:45.975649+0000 mgr.y (mgr.24475) 685 : cluster [DBG] pgmap v1112: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-02-18T01:34:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:47 smithi105 bash[19734]: audit 2024-02-18T01:34:46.608965+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:34:47.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:47 smithi105 bash[19734]: cluster 2024-02-18T01:34:46.609078+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T01:34:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:47 smithi105 bash[27669]: audit 2024-02-18T01:34:46.608965+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:34:47.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:47 smithi105 bash[27669]: cluster 2024-02-18T01:34:46.609078+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T01:34:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:47 smithi155 bash[20805]: audit 2024-02-18T01:34:46.608965+0000 mon.a (mon.0) 3017 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-128","var": "hit_set_type","val": "explicit_object"}]': finished 2024-02-18T01:34:47.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:47 smithi155 bash[20805]: cluster 2024-02-18T01:34:46.609078+0000 mon.a (mon.0) 3018 : cluster [DBG] osdmap e656: 8 total, 8 up, 8 in 2024-02-18T01:34:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[27669]: audit 2024-02-18T01:34:47.642658+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[27669]: audit 2024-02-18T01:34:47.643563+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]: dispatch 2024-02-18T01:34:48.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[27669]: cluster 2024-02-18T01:34:47.977847+0000 mgr.y (mgr.24475) 686 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[19734]: audit 2024-02-18T01:34:47.642658+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[19734]: audit 2024-02-18T01:34:47.643563+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]: dispatch 2024-02-18T01:34:48.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:48 smithi105 bash[19734]: cluster 2024-02-18T01:34:47.977847+0000 mgr.y (mgr.24475) 686 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:48 smithi155 bash[20805]: audit 2024-02-18T01:34:47.642658+0000 mon.a (mon.0) 3019 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:34:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:48 smithi155 bash[20805]: audit 2024-02-18T01:34:47.643563+0000 mon.a (mon.0) 3020 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]: dispatch 2024-02-18T01:34:48.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:48 smithi155 bash[20805]: cluster 2024-02-18T01:34:47.977847+0000 mgr.y (mgr.24475) 686 : cluster [DBG] pgmap v1114: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12102 ms) 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 00000010 69 73 |is| 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 00000012 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (13151 ms) 2024-02-18T01:34:49.645 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: require_osd_release = pacific 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6054 ms) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 43 tests from LibRadosTwoPoolsPP (496193 ms total) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1355 ms) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (10742 ms) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18161 ms) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (1 ms) 2024-02-18T01:34:49.646 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30259 ms total) 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7044 ms) 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8076 ms) 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: waiting for scrub... 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: done waiting 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24965 ms) 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9495 ms) 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-02-18T01:34:49.647 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8002 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8097 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10091 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7753 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12221 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8066 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13144 ms) 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-02-18T01:34:49.648 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7586 ms) 2024-02-18T01:34:49.649 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-02-18T01:34:49.649 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8068 ms) 2024-02-18T01:34:49.649 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-02-18T01:34:49.649 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: hmm, no HitSet yet 2024-02-18T01:34:49.649 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: ok, hit_set contains 248:602f83fe:::foo:head 2024-02-18T01:34:49.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[19734]: audit 2024-02-18T01:34:48.188407+0000 mgr.y (mgr.24475) 687 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:49.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[19734]: audit 2024-02-18T01:34:48.635171+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]': finished 2024-02-18T01:34:49.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[19734]: cluster 2024-02-18T01:34:48.635304+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T01:34:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[27669]: audit 2024-02-18T01:34:48.188407+0000 mgr.y (mgr.24475) 687 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[27669]: audit 2024-02-18T01:34:48.635171+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]': finished 2024-02-18T01:34:49.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:49 smithi105 bash[27669]: cluster 2024-02-18T01:34:48.635304+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T01:34:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:49 smithi155 bash[20805]: audit 2024-02-18T01:34:48.188407+0000 mgr.y (mgr.24475) 687 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:49 smithi155 bash[20805]: audit 2024-02-18T01:34:48.635171+0000 mon.a (mon.0) 3021 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-128"}]': finished 2024-02-18T01:34:49.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:49 smithi155 bash[20805]: cluster 2024-02-18T01:34:48.635304+0000 mon.a (mon.0) 3022 : cluster [DBG] osdmap e657: 8 total, 8 up, 8 in 2024-02-18T01:34:50.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:50 smithi105 bash[27669]: cluster 2024-02-18T01:34:49.642820+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T01:34:50.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:50 smithi105 bash[27669]: cluster 2024-02-18T01:34:49.978598+0000 mgr.y (mgr.24475) 688 : cluster [DBG] pgmap v1117: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:50.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:50 smithi105 bash[19734]: cluster 2024-02-18T01:34:49.642820+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T01:34:50.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:50 smithi105 bash[19734]: cluster 2024-02-18T01:34:49.978598+0000 mgr.y (mgr.24475) 688 : cluster [DBG] pgmap v1117: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:50 smithi155 bash[20805]: cluster 2024-02-18T01:34:49.642820+0000 mon.a (mon.0) 3023 : cluster [DBG] osdmap e658: 8 total, 8 up, 8 in 2024-02-18T01:34:50.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:50 smithi155 bash[20805]: cluster 2024-02-18T01:34:49.978598+0000 mgr.y (mgr.24475) 688 : cluster [DBG] pgmap v1117: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:34:51.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:51 smithi105 bash[19734]: cluster 2024-02-18T01:34:50.658971+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T01:34:51.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:51 smithi105 bash[19734]: audit 2024-02-18T01:34:50.660106+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:51.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:51 smithi105 bash[27669]: cluster 2024-02-18T01:34:50.658971+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T01:34:51.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:51 smithi105 bash[27669]: audit 2024-02-18T01:34:50.660106+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:51.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:51 smithi155 bash[20805]: cluster 2024-02-18T01:34:50.658971+0000 mon.a (mon.0) 3024 : cluster [DBG] osdmap e659: 8 total, 8 up, 8 in 2024-02-18T01:34:51.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:51 smithi155 bash[20805]: audit 2024-02-18T01:34:50.660106+0000 mon.a (mon.0) 3025 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:34:51.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:34:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:34:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[19734]: audit 2024-02-18T01:34:51.671594+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[19734]: cluster 2024-02-18T01:34:51.671779+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T01:34:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[19734]: audit 2024-02-18T01:34:51.674553+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:52.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[19734]: cluster 2024-02-18T01:34:51.979415+0000 mgr.y (mgr.24475) 689 : cluster [DBG] pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[27669]: audit 2024-02-18T01:34:51.671594+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[27669]: cluster 2024-02-18T01:34:51.671779+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T01:34:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[27669]: audit 2024-02-18T01:34:51.674553+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:52.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:52 smithi105 bash[27669]: cluster 2024-02-18T01:34:51.979415+0000 mgr.y (mgr.24475) 689 : cluster [DBG] pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:52 smithi155 bash[20805]: audit 2024-02-18T01:34:51.671594+0000 mon.a (mon.0) 3026 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:34:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:52 smithi155 bash[20805]: cluster 2024-02-18T01:34:51.671779+0000 mon.a (mon.0) 3027 : cluster [DBG] osdmap e660: 8 total, 8 up, 8 in 2024-02-18T01:34:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:52 smithi155 bash[20805]: audit 2024-02-18T01:34:51.674553+0000 mon.a (mon.0) 3028 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:34:52.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:52 smithi155 bash[20805]: cluster 2024-02-18T01:34:51.979415+0000 mgr.y (mgr.24475) 689 : cluster [DBG] pgmap v1120: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:34:53.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[27669]: audit 2024-02-18T01:34:52.669473+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:53.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[27669]: cluster 2024-02-18T01:34:52.669568+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T01:34:53.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[27669]: audit 2024-02-18T01:34:52.670922+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:34:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[19734]: audit 2024-02-18T01:34:52.669473+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[19734]: cluster 2024-02-18T01:34:52.669568+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T01:34:53.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:53 smithi105 bash[19734]: audit 2024-02-18T01:34:52.670922+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:34:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:53 smithi155 bash[20805]: audit 2024-02-18T01:34:52.669473+0000 mon.a (mon.0) 3029 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:34:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:53 smithi155 bash[20805]: cluster 2024-02-18T01:34:52.669568+0000 mon.a (mon.0) 3030 : cluster [DBG] osdmap e661: 8 total, 8 up, 8 in 2024-02-18T01:34:53.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:53 smithi155 bash[20805]: audit 2024-02-18T01:34:52.670922+0000 mon.a (mon.0) 3031 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]: dispatch 2024-02-18T01:34:54.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: audit 2024-02-18T01:34:53.675437+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: cluster 2024-02-18T01:34:53.675610+0000 mon.a (mon.0) 3033 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: audit 2024-02-18T01:34:53.677265+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: cluster 2024-02-18T01:34:53.980252+0000 mgr.y (mgr.24475) 690 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: audit 2024-02-18T01:34:54.679957+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: cluster 2024-02-18T01:34:54.680102+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[19734]: audit 2024-02-18T01:34:54.681532+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: audit 2024-02-18T01:34:53.675437+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: cluster 2024-02-18T01:34:53.675610+0000 mon.a (mon.0) 3033 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T01:34:54.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: audit 2024-02-18T01:34:53.677265+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:34:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: cluster 2024-02-18T01:34:53.980252+0000 mgr.y (mgr.24475) 690 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: audit 2024-02-18T01:34:54.679957+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:34:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: cluster 2024-02-18T01:34:54.680102+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T01:34:54.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:54 smithi105 bash[27669]: audit 2024-02-18T01:34:54.681532+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: audit 2024-02-18T01:34:53.675437+0000 mon.a (mon.0) 3032 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_count","val": "3"}]': finished 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: cluster 2024-02-18T01:34:53.675610+0000 mon.a (mon.0) 3033 : cluster [DBG] osdmap e662: 8 total, 8 up, 8 in 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: audit 2024-02-18T01:34:53.677265+0000 mon.a (mon.0) 3034 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]: dispatch 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: cluster 2024-02-18T01:34:53.980252+0000 mgr.y (mgr.24475) 690 : cluster [DBG] pgmap v1123: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: audit 2024-02-18T01:34:54.679957+0000 mon.a (mon.0) 3035 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_period","val": "3"}]': finished 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: cluster 2024-02-18T01:34:54.680102+0000 mon.a (mon.0) 3036 : cluster [DBG] osdmap e663: 8 total, 8 up, 8 in 2024-02-18T01:34:54.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:54 smithi155 bash[20805]: audit 2024-02-18T01:34:54.681532+0000 mon.a (mon.0) 3037 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:34:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:34:56] "GET /metrics HTTP/1.1" 200 191059 "" "Prometheus/2.33.4" 2024-02-18T01:34:56.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[19734]: audit 2024-02-18T01:34:55.681142+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[19734]: cluster 2024-02-18T01:34:55.681224+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[19734]: audit 2024-02-18T01:34:55.682110+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[19734]: cluster 2024-02-18T01:34:55.981094+0000 mgr.y (mgr.24475) 691 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[27669]: audit 2024-02-18T01:34:55.681142+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[27669]: cluster 2024-02-18T01:34:55.681224+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[27669]: audit 2024-02-18T01:34:55.682110+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:34:56.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:56 smithi105 bash[27669]: cluster 2024-02-18T01:34:55.981094+0000 mgr.y (mgr.24475) 691 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:56 smithi155 bash[20805]: audit 2024-02-18T01:34:55.681142+0000 mon.a (mon.0) 3038 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:34:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:56 smithi155 bash[20805]: cluster 2024-02-18T01:34:55.681224+0000 mon.a (mon.0) 3039 : cluster [DBG] osdmap e664: 8 total, 8 up, 8 in 2024-02-18T01:34:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:56 smithi155 bash[20805]: audit 2024-02-18T01:34:55.682110+0000 mon.a (mon.0) 3040 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-02-18T01:34:56.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:56 smithi155 bash[20805]: cluster 2024-02-18T01:34:55.981094+0000 mgr.y (mgr.24475) 691 : cluster [DBG] pgmap v1126: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:34:57.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:57 smithi105 bash[19734]: audit 2024-02-18T01:34:56.694069+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:34:57.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:57 smithi105 bash[19734]: cluster 2024-02-18T01:34:56.694274+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T01:34:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:57 smithi105 bash[27669]: audit 2024-02-18T01:34:56.694069+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:34:57.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:57 smithi105 bash[27669]: cluster 2024-02-18T01:34:56.694274+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T01:34:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:57 smithi155 bash[20805]: audit 2024-02-18T01:34:56.694069+0000 mon.a (mon.0) 3041 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-130","var": "hit_set_fpp","val": ".01"}]': finished 2024-02-18T01:34:57.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:57 smithi155 bash[20805]: cluster 2024-02-18T01:34:56.694274+0000 mon.a (mon.0) 3042 : cluster [DBG] osdmap e665: 8 total, 8 up, 8 in 2024-02-18T01:34:58.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:34:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:34:58.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:58 smithi105 bash[19734]: cluster 2024-02-18T01:34:57.983204+0000 mgr.y (mgr.24475) 692 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-02-18T01:34:58.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:58 smithi105 bash[27669]: cluster 2024-02-18T01:34:57.983204+0000 mgr.y (mgr.24475) 692 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-02-18T01:34:58.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:58 smithi155 bash[20805]: cluster 2024-02-18T01:34:57.983204+0000 mgr.y (mgr.24475) 692 : cluster [DBG] pgmap v1128: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 KiB/s wr, 1 op/s 2024-02-18T01:34:59.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:34:59 smithi105 bash[27669]: audit 2024-02-18T01:34:58.192117+0000 mgr.y (mgr.24475) 693 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:59.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:34:59 smithi105 bash[19734]: audit 2024-02-18T01:34:58.192117+0000 mgr.y (mgr.24475) 693 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:34:59.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:34:59 smithi155 bash[20805]: audit 2024-02-18T01:34:58.192117+0000 mgr.y (mgr.24475) 693 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:00.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:00 smithi105 bash[27669]: cluster 2024-02-18T01:34:59.984047+0000 mgr.y (mgr.24475) 694 : cluster [DBG] pgmap v1129: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-02-18T01:35:00.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:00 smithi105 bash[19734]: cluster 2024-02-18T01:34:59.984047+0000 mgr.y (mgr.24475) 694 : cluster [DBG] pgmap v1129: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-02-18T01:35:00.992 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:00 smithi155 bash[20805]: cluster 2024-02-18T01:34:59.984047+0000 mgr.y (mgr.24475) 694 : cluster [DBG] pgmap v1129: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 1.3 KiB/s wr, 0 op/s 2024-02-18T01:35:01.993 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:03 smithi105 bash[19734]: cluster 2024-02-18T01:35:01.984890+0000 mgr.y (mgr.24475) 695 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 700 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-02-18T01:35:03.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:03 smithi105 bash[27669]: cluster 2024-02-18T01:35:01.984890+0000 mgr.y (mgr.24475) 695 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 700 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-02-18T01:35:03.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:03 smithi155 bash[20805]: cluster 2024-02-18T01:35:01.984890+0000 mgr.y (mgr.24475) 695 : cluster [DBG] pgmap v1130: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 700 B/s rd, 1.1 KiB/s wr, 0 op/s 2024-02-18T01:35:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:05 smithi105 bash[19734]: cluster 2024-02-18T01:35:03.986976+0000 mgr.y (mgr.24475) 696 : cluster [DBG] pgmap v1131: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-02-18T01:35:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:05 smithi105 bash[27669]: cluster 2024-02-18T01:35:03.986976+0000 mgr.y (mgr.24475) 696 : cluster [DBG] pgmap v1131: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-02-18T01:35:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:05 smithi155 bash[20805]: cluster 2024-02-18T01:35:03.986976+0000 mgr.y (mgr.24475) 696 : cluster [DBG] pgmap v1131: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.9 KiB/s wr, 2 op/s 2024-02-18T01:35:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:06] "GET /metrics HTTP/1.1" 200 191097 "" "Prometheus/2.33.4" 2024-02-18T01:35:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:07 smithi105 bash[19734]: cluster 2024-02-18T01:35:05.987905+0000 mgr.y (mgr.24475) 697 : cluster [DBG] pgmap v1132: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:35:07.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:07 smithi105 bash[27669]: cluster 2024-02-18T01:35:05.987905+0000 mgr.y (mgr.24475) 697 : cluster [DBG] pgmap v1132: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:35:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:07 smithi155 bash[20805]: cluster 2024-02-18T01:35:05.987905+0000 mgr.y (mgr.24475) 697 : cluster [DBG] pgmap v1132: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:35:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[19734]: cluster 2024-02-18T01:35:07.990077+0000 mgr.y (mgr.24475) 698 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-02-18T01:35:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[19734]: audit 2024-02-18T01:35:08.768412+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[19734]: audit 2024-02-18T01:35:08.769184+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]: dispatch 2024-02-18T01:35:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[27669]: cluster 2024-02-18T01:35:07.990077+0000 mgr.y (mgr.24475) 698 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-02-18T01:35:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[27669]: audit 2024-02-18T01:35:08.768412+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:09 smithi105 bash[27669]: audit 2024-02-18T01:35:08.769184+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]: dispatch 2024-02-18T01:35:09.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:09 smithi155 bash[20805]: cluster 2024-02-18T01:35:07.990077+0000 mgr.y (mgr.24475) 698 : cluster [DBG] pgmap v1133: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 8.1 KiB/s wr, 2 op/s 2024-02-18T01:35:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:09 smithi155 bash[20805]: audit 2024-02-18T01:35:08.768412+0000 mon.a (mon.0) 3043 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:09 smithi155 bash[20805]: audit 2024-02-18T01:35:08.769184+0000 mon.a (mon.0) 3044 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]: dispatch 2024-02-18T01:35:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[27669]: audit 2024-02-18T01:35:08.203407+0000 mgr.y (mgr.24475) 699 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[27669]: audit 2024-02-18T01:35:09.077523+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]': finished 2024-02-18T01:35:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[27669]: cluster 2024-02-18T01:35:09.077651+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T01:35:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[19734]: audit 2024-02-18T01:35:08.203407+0000 mgr.y (mgr.24475) 699 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[19734]: audit 2024-02-18T01:35:09.077523+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]': finished 2024-02-18T01:35:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:10 smithi105 bash[19734]: cluster 2024-02-18T01:35:09.077651+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T01:35:10.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:10 smithi155 bash[20805]: audit 2024-02-18T01:35:08.203407+0000 mgr.y (mgr.24475) 699 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:10 smithi155 bash[20805]: audit 2024-02-18T01:35:09.077523+0000 mon.a (mon.0) 3045 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-130"}]': finished 2024-02-18T01:35:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:10 smithi155 bash[20805]: cluster 2024-02-18T01:35:09.077651+0000 mon.a (mon.0) 3046 : cluster [DBG] osdmap e666: 8 total, 8 up, 8 in 2024-02-18T01:35:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[19734]: cluster 2024-02-18T01:35:09.990908+0000 mgr.y (mgr.24475) 700 : cluster [DBG] pgmap v1135: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-18T01:35:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[19734]: cluster 2024-02-18T01:35:10.070911+0000 mon.a (mon.0) 3047 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[19734]: cluster 2024-02-18T01:35:10.083477+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T01:35:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[27669]: cluster 2024-02-18T01:35:09.990908+0000 mgr.y (mgr.24475) 700 : cluster [DBG] pgmap v1135: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-18T01:35:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[27669]: cluster 2024-02-18T01:35:10.070911+0000 mon.a (mon.0) 3047 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:11 smithi105 bash[27669]: cluster 2024-02-18T01:35:10.083477+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T01:35:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:11 smithi155 bash[20805]: cluster 2024-02-18T01:35:09.990908+0000 mgr.y (mgr.24475) 700 : cluster [DBG] pgmap v1135: 204 pgs: 204 active+clean; 3.9 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 8.3 KiB/s wr, 2 op/s 2024-02-18T01:35:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:11 smithi155 bash[20805]: cluster 2024-02-18T01:35:10.070911+0000 mon.a (mon.0) 3047 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:11 smithi155 bash[20805]: cluster 2024-02-18T01:35:10.083477+0000 mon.a (mon.0) 3048 : cluster [DBG] osdmap e667: 8 total, 8 up, 8 in 2024-02-18T01:35:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:12 smithi155 bash[20805]: cluster 2024-02-18T01:35:11.106129+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T01:35:12.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:12 smithi155 bash[20805]: audit 2024-02-18T01:35:11.114207+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:12.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:12 smithi105 bash[27669]: cluster 2024-02-18T01:35:11.106129+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T01:35:12.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:12 smithi105 bash[27669]: audit 2024-02-18T01:35:11.114207+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:12 smithi105 bash[19734]: cluster 2024-02-18T01:35:11.106129+0000 mon.a (mon.0) 3049 : cluster [DBG] osdmap e668: 8 total, 8 up, 8 in 2024-02-18T01:35:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:12 smithi105 bash[19734]: audit 2024-02-18T01:35:11.114207+0000 mon.a (mon.0) 3050 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:13.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[27669]: cluster 2024-02-18T01:35:11.991732+0000 mgr.y (mgr.24475) 701 : cluster [DBG] pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:13.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[27669]: audit 2024-02-18T01:35:12.118168+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:13.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[27669]: cluster 2024-02-18T01:35:12.118471+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T01:35:13.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[27669]: audit 2024-02-18T01:35:12.171852+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[19734]: cluster 2024-02-18T01:35:11.991732+0000 mgr.y (mgr.24475) 701 : cluster [DBG] pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[19734]: audit 2024-02-18T01:35:12.118168+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[19734]: cluster 2024-02-18T01:35:12.118471+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T01:35:13.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:13 smithi105 bash[19734]: audit 2024-02-18T01:35:12.171852+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:13 smithi155 bash[20805]: cluster 2024-02-18T01:35:11.991732+0000 mgr.y (mgr.24475) 701 : cluster [DBG] pgmap v1138: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:13 smithi155 bash[20805]: audit 2024-02-18T01:35:12.118168+0000 mon.a (mon.0) 3051 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:13 smithi155 bash[20805]: cluster 2024-02-18T01:35:12.118471+0000 mon.a (mon.0) 3052 : cluster [DBG] osdmap e669: 8 total, 8 up, 8 in 2024-02-18T01:35:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:13 smithi155 bash[20805]: audit 2024-02-18T01:35:12.171852+0000 mon.a (mon.0) 3053 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:14.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: audit 2024-02-18T01:35:13.118649+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: cluster 2024-02-18T01:35:13.118826+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: audit 2024-02-18T01:35:13.120446+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: cluster 2024-02-18T01:35:13.992569+0000 mgr.y (mgr.24475) 702 : cluster [DBG] pgmap v1141: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: audit 2024-02-18T01:35:14.121181+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[19734]: cluster 2024-02-18T01:35:14.121382+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: audit 2024-02-18T01:35:13.118649+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: cluster 2024-02-18T01:35:13.118826+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: audit 2024-02-18T01:35:13.120446+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:14.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: cluster 2024-02-18T01:35:13.992569+0000 mgr.y (mgr.24475) 702 : cluster [DBG] pgmap v1141: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T01:35:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: audit 2024-02-18T01:35:14.121181+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:14.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:14 smithi105 bash[27669]: cluster 2024-02-18T01:35:14.121382+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: audit 2024-02-18T01:35:13.118649+0000 mon.a (mon.0) 3054 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: cluster 2024-02-18T01:35:13.118826+0000 mon.a (mon.0) 3055 : cluster [DBG] osdmap e670: 8 total, 8 up, 8 in 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: audit 2024-02-18T01:35:13.120446+0000 mon.a (mon.0) 3056 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: cluster 2024-02-18T01:35:13.992569+0000 mgr.y (mgr.24475) 702 : cluster [DBG] pgmap v1141: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: audit 2024-02-18T01:35:14.121181+0000 mon.a (mon.0) 3057 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:14 smithi155 bash[20805]: cluster 2024-02-18T01:35:14.121382+0000 mon.a (mon.0) 3058 : cluster [DBG] osdmap e671: 8 total, 8 up, 8 in 2024-02-18T01:35:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[19734]: audit 2024-02-18T01:35:14.122074+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]: dispatch 2024-02-18T01:35:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[19734]: cluster 2024-02-18T01:35:15.115198+0000 mon.a (mon.0) 3060 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[19734]: audit 2024-02-18T01:35:15.123232+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]': finished 2024-02-18T01:35:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[19734]: cluster 2024-02-18T01:35:15.123356+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[19734]: audit 2024-02-18T01:35:15.123974+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[27669]: audit 2024-02-18T01:35:14.122074+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]: dispatch 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[27669]: cluster 2024-02-18T01:35:15.115198+0000 mon.a (mon.0) 3060 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[27669]: audit 2024-02-18T01:35:15.123232+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]': finished 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[27669]: cluster 2024-02-18T01:35:15.123356+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T01:35:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:15 smithi105 bash[27669]: audit 2024-02-18T01:35:15.123974+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:15 smithi155 bash[20805]: audit 2024-02-18T01:35:14.122074+0000 mon.a (mon.0) 3059 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]: dispatch 2024-02-18T01:35:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:15 smithi155 bash[20805]: cluster 2024-02-18T01:35:15.115198+0000 mon.a (mon.0) 3060 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:15 smithi155 bash[20805]: audit 2024-02-18T01:35:15.123232+0000 mon.a (mon.0) 3061 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-132", "mode": "writeback"}]': finished 2024-02-18T01:35:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:15 smithi155 bash[20805]: cluster 2024-02-18T01:35:15.123356+0000 mon.a (mon.0) 3062 : cluster [DBG] osdmap e672: 8 total, 8 up, 8 in 2024-02-18T01:35:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:15 smithi155 bash[20805]: audit 2024-02-18T01:35:15.123974+0000 mon.a (mon.0) 3063 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:16] "GET /metrics HTTP/1.1" 200 191097 "" "Prometheus/2.33.4" 2024-02-18T01:35:17.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[19734]: cluster 2024-02-18T01:35:15.993446+0000 mgr.y (mgr.24475) 703 : cluster [DBG] pgmap v1144: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:35:17.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[19734]: audit 2024-02-18T01:35:16.126790+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:17.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[19734]: cluster 2024-02-18T01:35:16.126985+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T01:35:17.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[19734]: audit 2024-02-18T01:35:16.127556+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:17.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[27669]: cluster 2024-02-18T01:35:15.993446+0000 mgr.y (mgr.24475) 703 : cluster [DBG] pgmap v1144: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:35:17.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[27669]: audit 2024-02-18T01:35:16.126790+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:17.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[27669]: cluster 2024-02-18T01:35:16.126985+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T01:35:17.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:17 smithi105 bash[27669]: audit 2024-02-18T01:35:16.127556+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:17.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:17 smithi155 bash[20805]: cluster 2024-02-18T01:35:15.993446+0000 mgr.y (mgr.24475) 703 : cluster [DBG] pgmap v1144: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-02-18T01:35:17.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:17 smithi155 bash[20805]: audit 2024-02-18T01:35:16.126790+0000 mon.a (mon.0) 3064 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:17.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:17 smithi155 bash[20805]: cluster 2024-02-18T01:35:16.126985+0000 mon.a (mon.0) 3065 : cluster [DBG] osdmap e673: 8 total, 8 up, 8 in 2024-02-18T01:35:17.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:17 smithi155 bash[20805]: audit 2024-02-18T01:35:16.127556+0000 mon.a (mon.0) 3066 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:18.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: audit 2024-02-18T01:35:17.128804+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:18.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: cluster 2024-02-18T01:35:17.129011+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T01:35:18.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: audit 2024-02-18T01:35:17.131304+0000 mon.a (mon.0) 3069 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: cluster 2024-02-18T01:35:17.995516+0000 mgr.y (mgr.24475) 704 : cluster [DBG] pgmap v1147: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: cluster 2024-02-18T01:35:18.122958+0000 mon.a (mon.0) 3070 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: audit 2024-02-18T01:35:18.131413+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: cluster 2024-02-18T01:35:18.131492+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[27669]: audit 2024-02-18T01:35:18.132300+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: audit 2024-02-18T01:35:17.128804+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: cluster 2024-02-18T01:35:17.129011+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: audit 2024-02-18T01:35:17.131304+0000 mon.a (mon.0) 3069 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: cluster 2024-02-18T01:35:17.995516+0000 mgr.y (mgr.24475) 704 : cluster [DBG] pgmap v1147: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: cluster 2024-02-18T01:35:18.122958+0000 mon.a (mon.0) 3070 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:18.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: audit 2024-02-18T01:35:18.131413+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:18.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: cluster 2024-02-18T01:35:18.131492+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T01:35:18.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:18 smithi105 bash[19734]: audit 2024-02-18T01:35:18.132300+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:18.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: audit 2024-02-18T01:35:17.128804+0000 mon.a (mon.0) 3067 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: cluster 2024-02-18T01:35:17.129011+0000 mon.a (mon.0) 3068 : cluster [DBG] osdmap e674: 8 total, 8 up, 8 in 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: audit 2024-02-18T01:35:17.131304+0000 mon.a (mon.0) 3069 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: cluster 2024-02-18T01:35:17.995516+0000 mgr.y (mgr.24475) 704 : cluster [DBG] pgmap v1147: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: cluster 2024-02-18T01:35:18.122958+0000 mon.a (mon.0) 3070 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: audit 2024-02-18T01:35:18.131413+0000 mon.a (mon.0) 3071 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: cluster 2024-02-18T01:35:18.131492+0000 mon.a (mon.0) 3072 : cluster [DBG] osdmap e675: 8 total, 8 up, 8 in 2024-02-18T01:35:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:18 smithi155 bash[20805]: audit 2024-02-18T01:35:18.132300+0000 mon.a (mon.0) 3073 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[19734]: audit 2024-02-18T01:35:18.209040+0000 mgr.y (mgr.24475) 705 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[19734]: audit 2024-02-18T01:35:19.130927+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[19734]: cluster 2024-02-18T01:35:19.131082+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T01:35:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[19734]: audit 2024-02-18T01:35:19.132722+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:35:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[27669]: audit 2024-02-18T01:35:18.209040+0000 mgr.y (mgr.24475) 705 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[27669]: audit 2024-02-18T01:35:19.130927+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[27669]: cluster 2024-02-18T01:35:19.131082+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T01:35:19.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:19 smithi105 bash[27669]: audit 2024-02-18T01:35:19.132722+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:35:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:19 smithi155 bash[20805]: audit 2024-02-18T01:35:18.209040+0000 mgr.y (mgr.24475) 705 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:19 smithi155 bash[20805]: audit 2024-02-18T01:35:19.130927+0000 mon.a (mon.0) 3074 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:19 smithi155 bash[20805]: cluster 2024-02-18T01:35:19.131082+0000 mon.a (mon.0) 3075 : cluster [DBG] osdmap e676: 8 total, 8 up, 8 in 2024-02-18T01:35:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:19 smithi155 bash[20805]: audit 2024-02-18T01:35:19.132722+0000 mon.a (mon.0) 3076 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-02-18T01:35:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[19734]: cluster 2024-02-18T01:35:19.501542+0000 mon.a (mon.0) 3077 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[19734]: cluster 2024-02-18T01:35:19.996462+0000 mgr.y (mgr.24475) 706 : cluster [DBG] pgmap v1150: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[19734]: audit 2024-02-18T01:35:20.139103+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[19734]: cluster 2024-02-18T01:35:20.139279+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[27669]: cluster 2024-02-18T01:35:19.501542+0000 mon.a (mon.0) 3077 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[27669]: cluster 2024-02-18T01:35:19.996462+0000 mgr.y (mgr.24475) 706 : cluster [DBG] pgmap v1150: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[27669]: audit 2024-02-18T01:35:20.139103+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:35:20.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:20 smithi105 bash[27669]: cluster 2024-02-18T01:35:20.139279+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T01:35:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:20 smithi155 bash[20805]: cluster 2024-02-18T01:35:19.501542+0000 mon.a (mon.0) 3077 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:20 smithi155 bash[20805]: cluster 2024-02-18T01:35:19.996462+0000 mgr.y (mgr.24475) 706 : cluster [DBG] pgmap v1150: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:20 smithi155 bash[20805]: audit 2024-02-18T01:35:20.139103+0000 mon.a (mon.0) 3078 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-02-18T01:35:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:20 smithi155 bash[20805]: cluster 2024-02-18T01:35:20.139279+0000 mon.a (mon.0) 3079 : cluster [DBG] osdmap e677: 8 total, 8 up, 8 in 2024-02-18T01:35:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[27669]: audit 2024-02-18T01:35:20.143638+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:35:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[27669]: audit 2024-02-18T01:35:21.140066+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:35:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[27669]: cluster 2024-02-18T01:35:21.140239+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T01:35:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[19734]: audit 2024-02-18T01:35:20.143638+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:35:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[19734]: audit 2024-02-18T01:35:21.140066+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:35:21.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:21 smithi105 bash[19734]: cluster 2024-02-18T01:35:21.140239+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T01:35:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:21 smithi155 bash[20805]: audit 2024-02-18T01:35:20.143638+0000 mon.a (mon.0) 3080 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-02-18T01:35:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:21 smithi155 bash[20805]: audit 2024-02-18T01:35:21.140066+0000 mon.a (mon.0) 3081 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-132","var": "hit_set_search_last_n","val": "1"}]': finished 2024-02-18T01:35:21.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:21 smithi155 bash[20805]: cluster 2024-02-18T01:35:21.140239+0000 mon.a (mon.0) 3082 : cluster [DBG] osdmap e678: 8 total, 8 up, 8 in 2024-02-18T01:35:22.158 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:22.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:22 smithi105 bash[19734]: audit 2024-02-18T01:35:21.189278+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:22.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:22 smithi105 bash[19734]: cluster 2024-02-18T01:35:21.997334+0000 mgr.y (mgr.24475) 707 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:22.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:22 smithi105 bash[27669]: audit 2024-02-18T01:35:21.189278+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:22.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:22 smithi105 bash[27669]: cluster 2024-02-18T01:35:21.997334+0000 mgr.y (mgr.24475) 707 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:22.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:22 smithi155 bash[20805]: audit 2024-02-18T01:35:21.189278+0000 mon.a (mon.0) 3083 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:22.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:22 smithi155 bash[20805]: cluster 2024-02-18T01:35:21.997334+0000 mgr.y (mgr.24475) 707 : cluster [DBG] pgmap v1153: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:23.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[19734]: audit 2024-02-18T01:35:22.171966+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:23.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[19734]: cluster 2024-02-18T01:35:22.172136+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T01:35:23.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[19734]: audit 2024-02-18T01:35:22.175886+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:23.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[27669]: audit 2024-02-18T01:35:22.171966+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:23.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[27669]: cluster 2024-02-18T01:35:22.172136+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T01:35:23.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:23 smithi105 bash[27669]: audit 2024-02-18T01:35:22.175886+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:23.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:23 smithi155 bash[20805]: audit 2024-02-18T01:35:22.171966+0000 mon.a (mon.0) 3084 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:23.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:23 smithi155 bash[20805]: cluster 2024-02-18T01:35:22.172136+0000 mon.a (mon.0) 3085 : cluster [DBG] osdmap e679: 8 total, 8 up, 8 in 2024-02-18T01:35:23.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:23 smithi155 bash[20805]: audit 2024-02-18T01:35:22.175886+0000 mon.a (mon.0) 3086 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[19734]: audit 2024-02-18T01:35:23.172614+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[19734]: cluster 2024-02-18T01:35:23.172696+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T01:35:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[19734]: audit 2024-02-18T01:35:23.205563+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[19734]: audit 2024-02-18T01:35:23.206093+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[19734]: cluster 2024-02-18T01:35:23.998182+0000 mgr.y (mgr.24475) 708 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[27669]: audit 2024-02-18T01:35:23.172614+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[27669]: cluster 2024-02-18T01:35:23.172696+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[27669]: audit 2024-02-18T01:35:23.205563+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[27669]: audit 2024-02-18T01:35:23.206093+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:24.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:24 smithi105 bash[27669]: cluster 2024-02-18T01:35:23.998182+0000 mgr.y (mgr.24475) 708 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:24 smithi155 bash[20805]: audit 2024-02-18T01:35:23.172614+0000 mon.a (mon.0) 3087 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]': finished 2024-02-18T01:35:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:24 smithi155 bash[20805]: cluster 2024-02-18T01:35:23.172696+0000 mon.a (mon.0) 3088 : cluster [DBG] osdmap e680: 8 total, 8 up, 8 in 2024-02-18T01:35:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:24 smithi155 bash[20805]: audit 2024-02-18T01:35:23.205563+0000 mon.a (mon.0) 3089 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:24 smithi155 bash[20805]: audit 2024-02-18T01:35:23.206093+0000 mon.a (mon.0) 3090 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-132"}]: dispatch 2024-02-18T01:35:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:24 smithi155 bash[20805]: cluster 2024-02-18T01:35:23.998182+0000 mgr.y (mgr.24475) 708 : cluster [DBG] pgmap v1156: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:25.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:25 smithi105 bash[19734]: cluster 2024-02-18T01:35:24.198177+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T01:35:25.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:25 smithi105 bash[19734]: cluster 2024-02-18T01:35:24.627928+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:25.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:25 smithi105 bash[27669]: cluster 2024-02-18T01:35:24.198177+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T01:35:25.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:25 smithi105 bash[27669]: cluster 2024-02-18T01:35:24.627928+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:25 smithi155 bash[20805]: cluster 2024-02-18T01:35:24.198177+0000 mon.a (mon.0) 3091 : cluster [DBG] osdmap e681: 8 total, 8 up, 8 in 2024-02-18T01:35:25.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:25 smithi155 bash[20805]: cluster 2024-02-18T01:35:24.627928+0000 mon.a (mon.0) 3092 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:26.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[27669]: cluster 2024-02-18T01:35:25.210110+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T01:35:26.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[27669]: audit 2024-02-18T01:35:25.214975+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:26.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[27669]: cluster 2024-02-18T01:35:25.999010+0000 mgr.y (mgr.24475) 709 : cluster [DBG] pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[19734]: cluster 2024-02-18T01:35:25.210110+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T01:35:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[19734]: audit 2024-02-18T01:35:25.214975+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:26.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[19734]: cluster 2024-02-18T01:35:25.999010+0000 mgr.y (mgr.24475) 709 : cluster [DBG] pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:26.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:26 smithi155 bash[20805]: cluster 2024-02-18T01:35:25.210110+0000 mon.a (mon.0) 3093 : cluster [DBG] osdmap e682: 8 total, 8 up, 8 in 2024-02-18T01:35:26.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:26 smithi155 bash[20805]: audit 2024-02-18T01:35:25.214975+0000 mon.a (mon.0) 3094 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:26.495 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:26 smithi155 bash[20805]: cluster 2024-02-18T01:35:25.999010+0000 mgr.y (mgr.24475) 709 : cluster [DBG] pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-02-18T01:35:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:26] "GET /metrics HTTP/1.1" 200 191179 "" "Prometheus/2.33.4" 2024-02-18T01:35:27.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[27669]: audit 2024-02-18T01:35:26.214357+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:27.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[27669]: cluster 2024-02-18T01:35:26.214776+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T01:35:27.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[27669]: audit 2024-02-18T01:35:26.228544+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[19734]: audit 2024-02-18T01:35:26.214357+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[19734]: cluster 2024-02-18T01:35:26.214776+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T01:35:27.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:27 smithi105 bash[19734]: audit 2024-02-18T01:35:26.228544+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:27 smithi155 bash[20805]: audit 2024-02-18T01:35:26.214357+0000 mon.a (mon.0) 3095 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:27 smithi155 bash[20805]: cluster 2024-02-18T01:35:26.214776+0000 mon.a (mon.0) 3096 : cluster [DBG] osdmap e683: 8 total, 8 up, 8 in 2024-02-18T01:35:27.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:27 smithi155 bash[20805]: audit 2024-02-18T01:35:26.228544+0000 mon.a (mon.0) 3097 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:28.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.237948+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: cluster 2024-02-18T01:35:27.238076+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T01:35:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.238733+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.452309+0000 mon.a (mon.0) 3101 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:35:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.750420+0000 mon.a (mon.0) 3102 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:35:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.751791+0000 mon.a (mon.0) 3103 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:35:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: audit 2024-02-18T01:35:27.760323+0000 mon.a (mon.0) 3104 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:28.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:28 smithi155 bash[20805]: cluster 2024-02-18T01:35:28.001062+0000 mgr.y (mgr.24475) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.237948+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: cluster 2024-02-18T01:35:27.238076+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.238733+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.452309+0000 mon.a (mon.0) 3101 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.750420+0000 mon.a (mon.0) 3102 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.751791+0000 mon.a (mon.0) 3103 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: audit 2024-02-18T01:35:27.760323+0000 mon.a (mon.0) 3104 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[19734]: cluster 2024-02-18T01:35:28.001062+0000 mgr.y (mgr.24475) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.237948+0000 mon.a (mon.0) 3098 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: cluster 2024-02-18T01:35:27.238076+0000 mon.a (mon.0) 3099 : cluster [DBG] osdmap e684: 8 total, 8 up, 8 in 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.238733+0000 mon.a (mon.0) 3100 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.452309+0000 mon.a (mon.0) 3101 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.750420+0000 mon.a (mon.0) 3102 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.751791+0000 mon.a (mon.0) 3103 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: audit 2024-02-18T01:35:27.760323+0000 mon.a (mon.0) 3104 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:28.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:28 smithi105 bash[27669]: cluster 2024-02-18T01:35:28.001062+0000 mgr.y (mgr.24475) 710 : cluster [DBG] pgmap v1162: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:29.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: audit 2024-02-18T01:35:28.216096+0000 mgr.y (mgr.24475) 711 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:29.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: audit 2024-02-18T01:35:28.239579+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:29.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: cluster 2024-02-18T01:35:28.239657+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: audit 2024-02-18T01:35:28.242589+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]: dispatch 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: cluster 2024-02-18T01:35:29.233983+0000 mon.a (mon.0) 3108 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[27669]: audit 2024-02-18T01:35:29.245032+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]': finished 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: audit 2024-02-18T01:35:28.216096+0000 mgr.y (mgr.24475) 711 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: audit 2024-02-18T01:35:28.239579+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: cluster 2024-02-18T01:35:28.239657+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: audit 2024-02-18T01:35:28.242589+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]: dispatch 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: cluster 2024-02-18T01:35:29.233983+0000 mon.a (mon.0) 3108 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:29.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:29 smithi105 bash[19734]: audit 2024-02-18T01:35:29.245032+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]': finished 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: audit 2024-02-18T01:35:28.216096+0000 mgr.y (mgr.24475) 711 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: audit 2024-02-18T01:35:28.239579+0000 mon.a (mon.0) 3105 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: cluster 2024-02-18T01:35:28.239657+0000 mon.a (mon.0) 3106 : cluster [DBG] osdmap e685: 8 total, 8 up, 8 in 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: audit 2024-02-18T01:35:28.242589+0000 mon.a (mon.0) 3107 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]: dispatch 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: cluster 2024-02-18T01:35:29.233983+0000 mon.a (mon.0) 3108 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:29.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:29 smithi155 bash[20805]: audit 2024-02-18T01:35:29.245032+0000 mon.a (mon.0) 3109 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-134", "mode": "readproxy"}]': finished 2024-02-18T01:35:30.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[27669]: cluster 2024-02-18T01:35:29.245238+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T01:35:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[27669]: cluster 2024-02-18T01:35:29.630177+0000 mon.a (mon.0) 3111 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:30.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[27669]: cluster 2024-02-18T01:35:30.001864+0000 mgr.y (mgr.24475) 712 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[19734]: cluster 2024-02-18T01:35:29.245238+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T01:35:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[19734]: cluster 2024-02-18T01:35:29.630177+0000 mon.a (mon.0) 3111 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:30.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:30 smithi105 bash[19734]: cluster 2024-02-18T01:35:30.001864+0000 mgr.y (mgr.24475) 712 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:30 smithi155 bash[20805]: cluster 2024-02-18T01:35:29.245238+0000 mon.a (mon.0) 3110 : cluster [DBG] osdmap e686: 8 total, 8 up, 8 in 2024-02-18T01:35:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:30 smithi155 bash[20805]: cluster 2024-02-18T01:35:29.630177+0000 mon.a (mon.0) 3111 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:35:30.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:30 smithi155 bash[20805]: cluster 2024-02-18T01:35:30.001864+0000 mgr.y (mgr.24475) 712 : cluster [DBG] pgmap v1165: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-02-18T01:35:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:33.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:33 smithi105 bash[19734]: cluster 2024-02-18T01:35:32.002695+0000 mgr.y (mgr.24475) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 176 B/s wr, 1 op/s 2024-02-18T01:35:33.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:33 smithi105 bash[27669]: cluster 2024-02-18T01:35:32.002695+0000 mgr.y (mgr.24475) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 176 B/s wr, 1 op/s 2024-02-18T01:35:33.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:33 smithi155 bash[20805]: cluster 2024-02-18T01:35:32.002695+0000 mgr.y (mgr.24475) 713 : cluster [DBG] pgmap v1166: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 883 B/s rd, 176 B/s wr, 1 op/s 2024-02-18T01:35:35.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:35 smithi105 bash[19734]: cluster 2024-02-18T01:35:34.004833+0000 mgr.y (mgr.24475) 714 : cluster [DBG] pgmap v1167: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-02-18T01:35:35.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:35 smithi105 bash[27669]: cluster 2024-02-18T01:35:34.004833+0000 mgr.y (mgr.24475) 714 : cluster [DBG] pgmap v1167: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-02-18T01:35:35.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:35 smithi155 bash[20805]: cluster 2024-02-18T01:35:34.004833+0000 mgr.y (mgr.24475) 714 : cluster [DBG] pgmap v1167: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 151 B/s wr, 1 op/s 2024-02-18T01:35:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:36] "GET /metrics HTTP/1.1" 200 191292 "" "Prometheus/2.33.4" 2024-02-18T01:35:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:37 smithi105 bash[19734]: cluster 2024-02-18T01:35:36.005692+0000 mgr.y (mgr.24475) 715 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:35:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:37 smithi105 bash[19734]: audit 2024-02-18T01:35:36.523142+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:37 smithi105 bash[27669]: cluster 2024-02-18T01:35:36.005692+0000 mgr.y (mgr.24475) 715 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:35:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:37 smithi105 bash[27669]: audit 2024-02-18T01:35:36.523142+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:37.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:37 smithi155 bash[20805]: cluster 2024-02-18T01:35:36.005692+0000 mgr.y (mgr.24475) 715 : cluster [DBG] pgmap v1168: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-02-18T01:35:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:37 smithi155 bash[20805]: audit 2024-02-18T01:35:36.523142+0000 mon.a (mon.0) 3112 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:39 smithi105 bash[19734]: cluster 2024-02-18T01:35:38.007707+0000 mgr.y (mgr.24475) 716 : cluster [DBG] pgmap v1169: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:35:39.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:39 smithi105 bash[27669]: cluster 2024-02-18T01:35:38.007707+0000 mgr.y (mgr.24475) 716 : cluster [DBG] pgmap v1169: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:35:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:39 smithi155 bash[20805]: cluster 2024-02-18T01:35:38.007707+0000 mgr.y (mgr.24475) 716 : cluster [DBG] pgmap v1169: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:35:40.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:40 smithi105 bash[27669]: audit 2024-02-18T01:35:38.227365+0000 mgr.y (mgr.24475) 717 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:40.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:40 smithi105 bash[27669]: audit 2024-02-18T01:35:39.509540+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:40.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:40 smithi105 bash[19734]: audit 2024-02-18T01:35:38.227365+0000 mgr.y (mgr.24475) 717 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:40.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:40 smithi105 bash[19734]: audit 2024-02-18T01:35:39.509540+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:40 smithi155 bash[20805]: audit 2024-02-18T01:35:38.227365+0000 mgr.y (mgr.24475) 717 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:40 smithi155 bash[20805]: audit 2024-02-18T01:35:39.509540+0000 mon.a (mon.0) 3113 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[19734]: cluster 2024-02-18T01:35:40.008656+0000 mgr.y (mgr.24475) 718 : cluster [DBG] pgmap v1170: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[19734]: audit 2024-02-18T01:35:40.092478+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[19734]: cluster 2024-02-18T01:35:40.092552+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T01:35:41.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[19734]: audit 2024-02-18T01:35:40.095286+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[27669]: cluster 2024-02-18T01:35:40.008656+0000 mgr.y (mgr.24475) 718 : cluster [DBG] pgmap v1170: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[27669]: audit 2024-02-18T01:35:40.092478+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[27669]: cluster 2024-02-18T01:35:40.092552+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T01:35:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:41 smithi105 bash[27669]: audit 2024-02-18T01:35:40.095286+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:41 smithi155 bash[20805]: cluster 2024-02-18T01:35:40.008656+0000 mgr.y (mgr.24475) 718 : cluster [DBG] pgmap v1170: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:41 smithi155 bash[20805]: audit 2024-02-18T01:35:40.092478+0000 mon.a (mon.0) 3114 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:35:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:41 smithi155 bash[20805]: cluster 2024-02-18T01:35:40.092552+0000 mon.a (mon.0) 3115 : cluster [DBG] osdmap e687: 8 total, 8 up, 8 in 2024-02-18T01:35:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:41 smithi155 bash[20805]: audit 2024-02-18T01:35:40.095286+0000 mon.a (mon.0) 3116 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[19734]: cluster 2024-02-18T01:35:41.088396+0000 mon.a (mon.0) 3117 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[19734]: audit 2024-02-18T01:35:41.096721+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:42.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[19734]: cluster 2024-02-18T01:35:41.096808+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[19734]: audit 2024-02-18T01:35:41.134529+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[19734]: audit 2024-02-18T01:35:41.135050+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[27669]: cluster 2024-02-18T01:35:41.088396+0000 mon.a (mon.0) 3117 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[27669]: audit 2024-02-18T01:35:41.096721+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[27669]: cluster 2024-02-18T01:35:41.096808+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[27669]: audit 2024-02-18T01:35:41.134529+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:42.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:42 smithi105 bash[27669]: audit 2024-02-18T01:35:41.135050+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:42 smithi155 bash[20805]: cluster 2024-02-18T01:35:41.088396+0000 mon.a (mon.0) 3117 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:42 smithi155 bash[20805]: audit 2024-02-18T01:35:41.096721+0000 mon.a (mon.0) 3118 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]': finished 2024-02-18T01:35:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:42 smithi155 bash[20805]: cluster 2024-02-18T01:35:41.096808+0000 mon.a (mon.0) 3119 : cluster [DBG] osdmap e688: 8 total, 8 up, 8 in 2024-02-18T01:35:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:42 smithi155 bash[20805]: audit 2024-02-18T01:35:41.134529+0000 mon.a (mon.0) 3120 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:35:42.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:42 smithi155 bash[20805]: audit 2024-02-18T01:35:41.135050+0000 mon.a (mon.0) 3121 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-134"}]: dispatch 2024-02-18T01:35:43.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[19734]: cluster 2024-02-18T01:35:42.009625+0000 mgr.y (mgr.24475) 719 : cluster [DBG] pgmap v1173: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:35:43.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[19734]: cluster 2024-02-18T01:35:42.114813+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[19734]: cluster 2024-02-18T01:35:43.114823+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[19734]: audit 2024-02-18T01:35:43.117890+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[27669]: cluster 2024-02-18T01:35:42.009625+0000 mgr.y (mgr.24475) 719 : cluster [DBG] pgmap v1173: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[27669]: cluster 2024-02-18T01:35:42.114813+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[27669]: cluster 2024-02-18T01:35:43.114823+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T01:35:43.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:43 smithi105 bash[27669]: audit 2024-02-18T01:35:43.117890+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:43 smithi155 bash[20805]: cluster 2024-02-18T01:35:42.009625+0000 mgr.y (mgr.24475) 719 : cluster [DBG] pgmap v1173: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:35:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:43 smithi155 bash[20805]: cluster 2024-02-18T01:35:42.114813+0000 mon.a (mon.0) 3122 : cluster [DBG] osdmap e689: 8 total, 8 up, 8 in 2024-02-18T01:35:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:43 smithi155 bash[20805]: cluster 2024-02-18T01:35:43.114823+0000 mon.a (mon.0) 3123 : cluster [DBG] osdmap e690: 8 total, 8 up, 8 in 2024-02-18T01:35:43.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:43 smithi155 bash[20805]: audit 2024-02-18T01:35:43.117890+0000 mon.a (mon.0) 3124 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:35:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[19734]: cluster 2024-02-18T01:35:44.010487+0000 mgr.y (mgr.24475) 720 : cluster [DBG] pgmap v1176: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[19734]: audit 2024-02-18T01:35:44.116335+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[19734]: cluster 2024-02-18T01:35:44.116561+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T01:35:45.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[19734]: audit 2024-02-18T01:35:44.145343+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[27669]: cluster 2024-02-18T01:35:44.010487+0000 mgr.y (mgr.24475) 720 : cluster [DBG] pgmap v1176: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[27669]: audit 2024-02-18T01:35:44.116335+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[27669]: cluster 2024-02-18T01:35:44.116561+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T01:35:45.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:45 smithi105 bash[27669]: audit 2024-02-18T01:35:44.145343+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:45 smithi155 bash[20805]: cluster 2024-02-18T01:35:44.010487+0000 mgr.y (mgr.24475) 720 : cluster [DBG] pgmap v1176: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:45 smithi155 bash[20805]: audit 2024-02-18T01:35:44.116335+0000 mon.a (mon.0) 3125 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:35:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:45 smithi155 bash[20805]: cluster 2024-02-18T01:35:44.116561+0000 mon.a (mon.0) 3126 : cluster [DBG] osdmap e691: 8 total, 8 up, 8 in 2024-02-18T01:35:45.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:45 smithi155 bash[20805]: audit 2024-02-18T01:35:44.145343+0000 mon.a (mon.0) 3127 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:35:46.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[19734]: audit 2024-02-18T01:35:45.119511+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:46.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[19734]: cluster 2024-02-18T01:35:45.119583+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T01:35:46.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[19734]: audit 2024-02-18T01:35:45.122934+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:35:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[27669]: audit 2024-02-18T01:35:45.119511+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[27669]: cluster 2024-02-18T01:35:45.119583+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T01:35:46.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[27669]: audit 2024-02-18T01:35:45.122934+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:35:46.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:46 smithi155 bash[20805]: audit 2024-02-18T01:35:45.119511+0000 mon.a (mon.0) 3128 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:35:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:46 smithi155 bash[20805]: cluster 2024-02-18T01:35:45.119583+0000 mon.a (mon.0) 3129 : cluster [DBG] osdmap e692: 8 total, 8 up, 8 in 2024-02-18T01:35:46.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:46 smithi155 bash[20805]: audit 2024-02-18T01:35:45.122934+0000 mon.a (mon.0) 3130 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:35:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:46] "GET /metrics HTTP/1.1" 200 191292 "" "Prometheus/2.33.4" 2024-02-18T01:35:47.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[19734]: cluster 2024-02-18T01:35:46.011267+0000 mgr.y (mgr.24475) 721 : cluster [DBG] pgmap v1179: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[19734]: audit 2024-02-18T01:35:46.123830+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[19734]: cluster 2024-02-18T01:35:46.124032+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[19734]: audit 2024-02-18T01:35:46.125943+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]: dispatch 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[19734]: cluster 2024-02-18T01:35:47.116892+0000 mon.a (mon.0) 3134 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[27669]: cluster 2024-02-18T01:35:46.011267+0000 mgr.y (mgr.24475) 721 : cluster [DBG] pgmap v1179: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[27669]: audit 2024-02-18T01:35:46.123830+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[27669]: cluster 2024-02-18T01:35:46.124032+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[27669]: audit 2024-02-18T01:35:46.125943+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]: dispatch 2024-02-18T01:35:47.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:47 smithi105 bash[27669]: cluster 2024-02-18T01:35:47.116892+0000 mon.a (mon.0) 3134 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:47 smithi155 bash[20805]: cluster 2024-02-18T01:35:46.011267+0000 mgr.y (mgr.24475) 721 : cluster [DBG] pgmap v1179: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:35:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:47 smithi155 bash[20805]: audit 2024-02-18T01:35:46.123830+0000 mon.a (mon.0) 3131 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi105-61833-101", "overlaypool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:35:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:47 smithi155 bash[20805]: cluster 2024-02-18T01:35:46.124032+0000 mon.a (mon.0) 3132 : cluster [DBG] osdmap e693: 8 total, 8 up, 8 in 2024-02-18T01:35:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:47 smithi155 bash[20805]: audit 2024-02-18T01:35:46.125943+0000 mon.a (mon.0) 3133 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]: dispatch 2024-02-18T01:35:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:47 smithi155 bash[20805]: cluster 2024-02-18T01:35:47.116892+0000 mon.a (mon.0) 3134 : cluster [WRN] Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-02-18T01:35:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[19734]: audit 2024-02-18T01:35:47.126015+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]': finished 2024-02-18T01:35:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[19734]: cluster 2024-02-18T01:35:47.126126+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T01:35:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[19734]: audit 2024-02-18T01:35:47.172438+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[19734]: cluster 2024-02-18T01:35:48.013181+0000 mgr.y (mgr.24475) 722 : cluster [DBG] pgmap v1182: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[27669]: audit 2024-02-18T01:35:47.126015+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]': finished 2024-02-18T01:35:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[27669]: cluster 2024-02-18T01:35:47.126126+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T01:35:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[27669]: audit 2024-02-18T01:35:47.172438+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:48.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:48 smithi105 bash[27669]: cluster 2024-02-18T01:35:48.013181+0000 mgr.y (mgr.24475) 722 : cluster [DBG] pgmap v1182: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:48 smithi155 bash[20805]: audit 2024-02-18T01:35:47.126015+0000 mon.a (mon.0) 3135 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi105-61833-136", "mode": "writeback"}]': finished 2024-02-18T01:35:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:48 smithi155 bash[20805]: cluster 2024-02-18T01:35:47.126126+0000 mon.a (mon.0) 3136 : cluster [DBG] osdmap e694: 8 total, 8 up, 8 in 2024-02-18T01:35:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:48 smithi155 bash[20805]: audit 2024-02-18T01:35:47.172438+0000 mon.a (mon.0) 3137 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]: dispatch 2024-02-18T01:35:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:48 smithi155 bash[20805]: cluster 2024-02-18T01:35:48.013181+0000 mgr.y (mgr.24475) 722 : cluster [DBG] pgmap v1182: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[19734]: audit 2024-02-18T01:35:48.147646+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[19734]: cluster 2024-02-18T01:35:48.147771+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T01:35:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[19734]: audit 2024-02-18T01:35:48.148902+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:49.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[19734]: audit 2024-02-18T01:35:48.237286+0000 mgr.y (mgr.24475) 723 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[27669]: audit 2024-02-18T01:35:48.147646+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[27669]: cluster 2024-02-18T01:35:48.147771+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T01:35:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[27669]: audit 2024-02-18T01:35:48.148902+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:49.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:49 smithi105 bash[27669]: audit 2024-02-18T01:35:48.237286+0000 mgr.y (mgr.24475) 723 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:49 smithi155 bash[20805]: audit 2024-02-18T01:35:48.147646+0000 mon.a (mon.0) 3138 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_count","val": "2"}]': finished 2024-02-18T01:35:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:49 smithi155 bash[20805]: cluster 2024-02-18T01:35:48.147771+0000 mon.a (mon.0) 3139 : cluster [DBG] osdmap e695: 8 total, 8 up, 8 in 2024-02-18T01:35:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:49 smithi155 bash[20805]: audit 2024-02-18T01:35:48.148902+0000 mon.a (mon.0) 3140 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]: dispatch 2024-02-18T01:35:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:49 smithi155 bash[20805]: audit 2024-02-18T01:35:48.237286+0000 mgr.y (mgr.24475) 723 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:35:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: audit 2024-02-18T01:35:49.152238+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: cluster 2024-02-18T01:35:49.152333+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: audit 2024-02-18T01:35:49.154700+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: cluster 2024-02-18T01:35:50.013944+0000 mgr.y (mgr.24475) 724 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: cluster 2024-02-18T01:35:50.145624+0000 mon.a (mon.0) 3144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: audit 2024-02-18T01:35:50.154040+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: cluster 2024-02-18T01:35:50.154164+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[19734]: audit 2024-02-18T01:35:50.155982+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: audit 2024-02-18T01:35:49.152238+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: cluster 2024-02-18T01:35:49.152333+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: audit 2024-02-18T01:35:49.154700+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:50.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: cluster 2024-02-18T01:35:50.013944+0000 mgr.y (mgr.24475) 724 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: cluster 2024-02-18T01:35:50.145624+0000 mon.a (mon.0) 3144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: audit 2024-02-18T01:35:50.154040+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: cluster 2024-02-18T01:35:50.154164+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T01:35:50.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:50 smithi105 bash[27669]: audit 2024-02-18T01:35:50.155982+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: audit 2024-02-18T01:35:49.152238+0000 mon.a (mon.0) 3141 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_period","val": "600"}]': finished 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: cluster 2024-02-18T01:35:49.152333+0000 mon.a (mon.0) 3142 : cluster [DBG] osdmap e696: 8 total, 8 up, 8 in 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: audit 2024-02-18T01:35:49.154700+0000 mon.a (mon.0) 3143 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]: dispatch 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: cluster 2024-02-18T01:35:50.013944+0000 mgr.y (mgr.24475) 724 : cluster [DBG] pgmap v1185: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: cluster 2024-02-18T01:35:50.145624+0000 mon.a (mon.0) 3144 : cluster [INF] Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: audit 2024-02-18T01:35:50.154040+0000 mon.a (mon.0) 3145 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "hit_set_type","val": "bloom"}]': finished 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: cluster 2024-02-18T01:35:50.154164+0000 mon.a (mon.0) 3146 : cluster [DBG] osdmap e697: 8 total, 8 up, 8 in 2024-02-18T01:35:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:50 smithi155 bash[20805]: audit 2024-02-18T01:35:50.155982+0000 mon.a (mon.0) 3147 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-02-18T01:35:52.163 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:35:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:35:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[27669]: audit 2024-02-18T01:35:51.156986+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[27669]: cluster 2024-02-18T01:35:51.157214+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T01:35:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[27669]: audit 2024-02-18T01:35:51.158824+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:35:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[27669]: audit 2024-02-18T01:35:51.523422+0000 mon.a (mon.0) 3151 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[27669]: cluster 2024-02-18T01:35:52.014701+0000 mgr.y (mgr.24475) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[19734]: audit 2024-02-18T01:35:51.156986+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[19734]: cluster 2024-02-18T01:35:51.157214+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[19734]: audit 2024-02-18T01:35:51.158824+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[19734]: audit 2024-02-18T01:35:51.523422+0000 mon.a (mon.0) 3151 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:52.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:52 smithi105 bash[19734]: cluster 2024-02-18T01:35:52.014701+0000 mgr.y (mgr.24475) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:52 smithi155 bash[20805]: audit 2024-02-18T01:35:51.156986+0000 mon.a (mon.0) 3148 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-02-18T01:35:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:52 smithi155 bash[20805]: cluster 2024-02-18T01:35:51.157214+0000 mon.a (mon.0) 3149 : cluster [DBG] osdmap e698: 8 total, 8 up, 8 in 2024-02-18T01:35:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:52 smithi155 bash[20805]: audit 2024-02-18T01:35:51.158824+0000 mon.a (mon.0) 3150 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]: dispatch 2024-02-18T01:35:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:52 smithi155 bash[20805]: audit 2024-02-18T01:35:51.523422+0000 mon.a (mon.0) 3151 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:35:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:52 smithi155 bash[20805]: cluster 2024-02-18T01:35:52.014701+0000 mgr.y (mgr.24475) 725 : cluster [DBG] pgmap v1188: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:35:53.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:53 smithi105 bash[19734]: audit 2024-02-18T01:35:52.157676+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:35:53.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:53 smithi105 bash[19734]: cluster 2024-02-18T01:35:52.158404+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T01:35:53.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:53 smithi105 bash[27669]: audit 2024-02-18T01:35:52.157676+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:35:53.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:53 smithi105 bash[27669]: cluster 2024-02-18T01:35:52.158404+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T01:35:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:53 smithi155 bash[20805]: audit 2024-02-18T01:35:52.157676+0000 mon.a (mon.0) 3152 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-136","var": "target_max_objects","val": "1"}]': finished 2024-02-18T01:35:53.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:53 smithi155 bash[20805]: cluster 2024-02-18T01:35:52.158404+0000 mon.a (mon.0) 3153 : cluster [DBG] osdmap e699: 8 total, 8 up, 8 in 2024-02-18T01:35:54.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:54 smithi105 bash[19734]: cluster 2024-02-18T01:35:54.016817+0000 mgr.y (mgr.24475) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:54.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:54 smithi105 bash[27669]: cluster 2024-02-18T01:35:54.016817+0000 mgr.y (mgr.24475) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:54.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:54 smithi155 bash[20805]: cluster 2024-02-18T01:35:54.016817+0000 mgr.y (mgr.24475) 726 : cluster [DBG] pgmap v1190: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-02-18T01:35:55.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:55 smithi105 bash[19734]: cluster 2024-02-18T01:35:54.164847+0000 mon.a (mon.0) 3154 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:35:55.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:55 smithi105 bash[27669]: cluster 2024-02-18T01:35:54.164847+0000 mon.a (mon.0) 3154 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:35:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:55 smithi155 bash[20805]: cluster 2024-02-18T01:35:54.164847+0000 mon.a (mon.0) 3154 : cluster [WRN] Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-02-18T01:35:56.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:56 smithi105 bash[27669]: cluster 2024-02-18T01:35:56.017726+0000 mgr.y (mgr.24475) 727 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:56.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:56 smithi105 bash[19734]: cluster 2024-02-18T01:35:56.017726+0000 mgr.y (mgr.24475) 727 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:56.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:56 smithi155 bash[20805]: cluster 2024-02-18T01:35:56.017726+0000 mgr.y (mgr.24475) 727 : cluster [DBG] pgmap v1191: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:35:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:35:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:35:56] "GET /metrics HTTP/1.1" 200 191452 "" "Prometheus/2.33.4" 2024-02-18T01:35:58.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:35:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:35:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:35:59 smithi105 bash[19734]: cluster 2024-02-18T01:35:58.019822+0000 mgr.y (mgr.24475) 728 : cluster [DBG] pgmap v1192: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:35:59.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:35:59 smithi105 bash[27669]: cluster 2024-02-18T01:35:58.019822+0000 mgr.y (mgr.24475) 728 : cluster [DBG] pgmap v1192: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:35:59.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:35:59 smithi155 bash[20805]: cluster 2024-02-18T01:35:58.019822+0000 mgr.y (mgr.24475) 728 : cluster [DBG] pgmap v1192: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:00.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:00 smithi105 bash[27669]: audit 2024-02-18T01:35:58.248593+0000 mgr.y (mgr.24475) 729 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:00 smithi105 bash[19734]: audit 2024-02-18T01:35:58.248593+0000 mgr.y (mgr.24475) 729 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:00 smithi155 bash[20805]: audit 2024-02-18T01:35:58.248593+0000 mgr.y (mgr.24475) 729 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:01 smithi105 bash[19734]: cluster 2024-02-18T01:36:00.020682+0000 mgr.y (mgr.24475) 730 : cluster [DBG] pgmap v1193: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:01.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:01 smithi105 bash[27669]: cluster 2024-02-18T01:36:00.020682+0000 mgr.y (mgr.24475) 730 : cluster [DBG] pgmap v1193: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:01.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:01 smithi155 bash[20805]: cluster 2024-02-18T01:36:00.020682+0000 mgr.y (mgr.24475) 730 : cluster [DBG] pgmap v1193: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:03 smithi105 bash[19734]: cluster 2024-02-18T01:36:02.021608+0000 mgr.y (mgr.24475) 731 : cluster [DBG] pgmap v1194: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:03 smithi105 bash[19734]: audit 2024-02-18T01:36:02.186605+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:03 smithi105 bash[27669]: cluster 2024-02-18T01:36:02.021608+0000 mgr.y (mgr.24475) 731 : cluster [DBG] pgmap v1194: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:03.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:03 smithi105 bash[27669]: audit 2024-02-18T01:36:02.186605+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:03 smithi155 bash[20805]: cluster 2024-02-18T01:36:02.021608+0000 mgr.y (mgr.24475) 731 : cluster [DBG] pgmap v1194: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:03 smithi155 bash[20805]: audit 2024-02-18T01:36:02.186605+0000 mon.a (mon.0) 3155 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[19734]: audit 2024-02-18T01:36:03.105732+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[19734]: cluster 2024-02-18T01:36:03.105908+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T01:36:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[19734]: audit 2024-02-18T01:36:03.107987+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[19734]: audit 2024-02-18T01:36:04.105811+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[19734]: cluster 2024-02-18T01:36:04.105920+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[27669]: audit 2024-02-18T01:36:03.105732+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[27669]: cluster 2024-02-18T01:36:03.105908+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[27669]: audit 2024-02-18T01:36:03.107987+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[27669]: audit 2024-02-18T01:36:04.105811+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:36:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:04 smithi105 bash[27669]: cluster 2024-02-18T01:36:04.105920+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T01:36:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:04 smithi155 bash[20805]: audit 2024-02-18T01:36:03.105732+0000 mon.a (mon.0) 3156 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:04 smithi155 bash[20805]: cluster 2024-02-18T01:36:03.105908+0000 mon.a (mon.0) 3157 : cluster [DBG] osdmap e700: 8 total, 8 up, 8 in 2024-02-18T01:36:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:04 smithi155 bash[20805]: audit 2024-02-18T01:36:03.107987+0000 mon.a (mon.0) 3158 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:04 smithi155 bash[20805]: audit 2024-02-18T01:36:04.105811+0000 mon.a (mon.0) 3159 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]': finished 2024-02-18T01:36:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:04 smithi155 bash[20805]: cluster 2024-02-18T01:36:04.105920+0000 mon.a (mon.0) 3160 : cluster [DBG] osdmap e701: 8 total, 8 up, 8 in 2024-02-18T01:36:05.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[27669]: cluster 2024-02-18T01:36:04.022494+0000 mgr.y (mgr.24475) 732 : cluster [DBG] pgmap v1196: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:05.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[27669]: audit 2024-02-18T01:36:04.141414+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:05.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[27669]: audit 2024-02-18T01:36:04.141917+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[19734]: cluster 2024-02-18T01:36:04.022494+0000 mgr.y (mgr.24475) 732 : cluster [DBG] pgmap v1196: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[19734]: audit 2024-02-18T01:36:04.141414+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:05.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:05 smithi105 bash[19734]: audit 2024-02-18T01:36:04.141917+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:05 smithi155 bash[20805]: cluster 2024-02-18T01:36:04.022494+0000 mgr.y (mgr.24475) 732 : cluster [DBG] pgmap v1196: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-02-18T01:36:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:05 smithi155 bash[20805]: audit 2024-02-18T01:36:04.141414+0000 mon.a (mon.0) 3161 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:05 smithi155 bash[20805]: audit 2024-02-18T01:36:04.141917+0000 mon.a (mon.0) 3162 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-136"}]: dispatch 2024-02-18T01:36:06.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:06 smithi105 bash[19734]: cluster 2024-02-18T01:36:05.132980+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T01:36:06.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:06 smithi105 bash[27669]: cluster 2024-02-18T01:36:05.132980+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T01:36:06.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:06 smithi155 bash[20805]: cluster 2024-02-18T01:36:05.132980+0000 mon.a (mon.0) 3163 : cluster [DBG] osdmap e702: 8 total, 8 up, 8 in 2024-02-18T01:36:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:06] "GET /metrics HTTP/1.1" 200 191538 "" "Prometheus/2.33.4" 2024-02-18T01:36:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[19734]: cluster 2024-02-18T01:36:06.023280+0000 mgr.y (mgr.24475) 733 : cluster [DBG] pgmap v1199: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-18T01:36:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[19734]: cluster 2024-02-18T01:36:06.128444+0000 mon.a (mon.0) 3164 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:36:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[19734]: cluster 2024-02-18T01:36:06.147715+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T01:36:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[19734]: audit 2024-02-18T01:36:06.150657+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[27669]: cluster 2024-02-18T01:36:06.023280+0000 mgr.y (mgr.24475) 733 : cluster [DBG] pgmap v1199: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-18T01:36:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[27669]: cluster 2024-02-18T01:36:06.128444+0000 mon.a (mon.0) 3164 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:36:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[27669]: cluster 2024-02-18T01:36:06.147715+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T01:36:07.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:07 smithi105 bash[27669]: audit 2024-02-18T01:36:06.150657+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:07 smithi155 bash[20805]: cluster 2024-02-18T01:36:06.023280+0000 mgr.y (mgr.24475) 733 : cluster [DBG] pgmap v1199: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-02-18T01:36:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:07 smithi155 bash[20805]: cluster 2024-02-18T01:36:06.128444+0000 mon.a (mon.0) 3164 : cluster [INF] Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-02-18T01:36:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:07 smithi155 bash[20805]: cluster 2024-02-18T01:36:06.147715+0000 mon.a (mon.0) 3165 : cluster [DBG] osdmap e703: 8 total, 8 up, 8 in 2024-02-18T01:36:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:07 smithi155 bash[20805]: audit 2024-02-18T01:36:06.150657+0000 mon.a (mon.0) 3166 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:08.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[19734]: audit 2024-02-18T01:36:07.148849+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[19734]: cluster 2024-02-18T01:36:07.149062+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T01:36:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[19734]: audit 2024-02-18T01:36:07.180382+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:36:08.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[19734]: cluster 2024-02-18T01:36:08.025244+0000 mgr.y (mgr.24475) 734 : cluster [DBG] pgmap v1202: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[27669]: audit 2024-02-18T01:36:07.148849+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[27669]: cluster 2024-02-18T01:36:07.149062+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T01:36:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[27669]: audit 2024-02-18T01:36:07.180382+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:36:08.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:08 smithi105 bash[27669]: cluster 2024-02-18T01:36:08.025244+0000 mgr.y (mgr.24475) 734 : cluster [DBG] pgmap v1202: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:08 smithi155 bash[20805]: audit 2024-02-18T01:36:07.148849+0000 mon.a (mon.0) 3167 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:08 smithi155 bash[20805]: cluster 2024-02-18T01:36:07.149062+0000 mon.a (mon.0) 3168 : cluster [DBG] osdmap e704: 8 total, 8 up, 8 in 2024-02-18T01:36:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:08 smithi155 bash[20805]: audit 2024-02-18T01:36:07.180382+0000 mon.a (mon.0) 3169 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-02-18T01:36:08.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:08 smithi155 bash[20805]: cluster 2024-02-18T01:36:08.025244+0000 mgr.y (mgr.24475) 734 : cluster [DBG] pgmap v1202: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[19734]: audit 2024-02-18T01:36:08.159289+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:36:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[19734]: cluster 2024-02-18T01:36:08.159373+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T01:36:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[19734]: audit 2024-02-18T01:36:08.165419+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:09.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[19734]: audit 2024-02-18T01:36:08.260037+0000 mgr.y (mgr.24475) 735 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[27669]: audit 2024-02-18T01:36:08.159289+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:36:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[27669]: cluster 2024-02-18T01:36:08.159373+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T01:36:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[27669]: audit 2024-02-18T01:36:08.165419+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:09 smithi105 bash[27669]: audit 2024-02-18T01:36:08.260037+0000 mgr.y (mgr.24475) 735 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:09 smithi155 bash[20805]: audit 2024-02-18T01:36:08.159289+0000 mon.a (mon.0) 3170 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138", "force_nonempty": "--force-nonempty" }]': finished 2024-02-18T01:36:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:09 smithi155 bash[20805]: cluster 2024-02-18T01:36:08.159373+0000 mon.a (mon.0) 3171 : cluster [DBG] osdmap e705: 8 total, 8 up, 8 in 2024-02-18T01:36:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:09 smithi155 bash[20805]: audit 2024-02-18T01:36:08.165419+0000 mon.a (mon.0) 3172 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:09 smithi155 bash[20805]: audit 2024-02-18T01:36:08.260037+0000 mgr.y (mgr.24475) 735 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[27669]: audit 2024-02-18T01:36:09.163586+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]': finished 2024-02-18T01:36:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[27669]: cluster 2024-02-18T01:36:09.163826+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T01:36:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[27669]: audit 2024-02-18T01:36:09.201909+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[27669]: audit 2024-02-18T01:36:09.202457+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:10.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[27669]: cluster 2024-02-18T01:36:10.026039+0000 mgr.y (mgr.24475) 736 : cluster [DBG] pgmap v1205: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:36:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[19734]: audit 2024-02-18T01:36:09.163586+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]': finished 2024-02-18T01:36:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[19734]: cluster 2024-02-18T01:36:09.163826+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T01:36:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[19734]: audit 2024-02-18T01:36:09.201909+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[19734]: audit 2024-02-18T01:36:09.202457+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:10.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:10 smithi105 bash[19734]: cluster 2024-02-18T01:36:10.026039+0000 mgr.y (mgr.24475) 736 : cluster [DBG] pgmap v1205: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:36:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:10 smithi155 bash[20805]: audit 2024-02-18T01:36:09.163586+0000 mon.a (mon.0) 3173 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]': finished 2024-02-18T01:36:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:10 smithi155 bash[20805]: cluster 2024-02-18T01:36:09.163826+0000 mon.a (mon.0) 3174 : cluster [DBG] osdmap e706: 8 total, 8 up, 8 in 2024-02-18T01:36:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:10 smithi155 bash[20805]: audit 2024-02-18T01:36:09.201909+0000 mon.a (mon.0) 3175 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:10 smithi155 bash[20805]: audit 2024-02-18T01:36:09.202457+0000 mon.a (mon.0) 3176 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-138"}]: dispatch 2024-02-18T01:36:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:10 smithi155 bash[20805]: cluster 2024-02-18T01:36:10.026039+0000 mgr.y (mgr.24475) 736 : cluster [DBG] pgmap v1205: 204 pgs: 204 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-02-18T01:36:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:11 smithi105 bash[19734]: cluster 2024-02-18T01:36:10.167344+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:11 smithi105 bash[19734]: cluster 2024-02-18T01:36:10.183650+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T01:36:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:11 smithi105 bash[27669]: cluster 2024-02-18T01:36:10.167344+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:11.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:11 smithi105 bash[27669]: cluster 2024-02-18T01:36:10.183650+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T01:36:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:11 smithi155 bash[20805]: cluster 2024-02-18T01:36:10.167344+0000 mon.a (mon.0) 3177 : cluster [WRN] Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:11.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:11 smithi155 bash[20805]: cluster 2024-02-18T01:36:10.183650+0000 mon.a (mon.0) 3178 : cluster [DBG] osdmap e707: 8 total, 8 up, 8 in 2024-02-18T01:36:12.197 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[19734]: cluster 2024-02-18T01:36:11.185996+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T01:36:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[19734]: audit 2024-02-18T01:36:11.193816+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[19734]: cluster 2024-02-18T01:36:12.026852+0000 mgr.y (mgr.24475) 737 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:12.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[27669]: cluster 2024-02-18T01:36:11.185996+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T01:36:12.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[27669]: audit 2024-02-18T01:36:11.193816+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:12.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:12 smithi105 bash[27669]: cluster 2024-02-18T01:36:12.026852+0000 mgr.y (mgr.24475) 737 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:12 smithi155 bash[20805]: cluster 2024-02-18T01:36:11.185996+0000 mon.a (mon.0) 3179 : cluster [DBG] osdmap e708: 8 total, 8 up, 8 in 2024-02-18T01:36:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:12 smithi155 bash[20805]: audit 2024-02-18T01:36:11.193816+0000 mon.a (mon.0) 3180 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:12 smithi155 bash[20805]: cluster 2024-02-18T01:36:12.026852+0000 mgr.y (mgr.24475) 737 : cluster [DBG] pgmap v1208: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:13.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:13 smithi155 bash[20805]: audit 2024-02-18T01:36:12.189755+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:13.649 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:13 smithi155 bash[20805]: cluster 2024-02-18T01:36:12.189905+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T01:36:13.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[27669]: audit 2024-02-18T01:36:12.189755+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:13.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[27669]: cluster 2024-02-18T01:36:12.189905+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T01:36:13.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[27669]: audit 2024-02-18T01:36:12.231828+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:13.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[27669]: audit 2024-02-18T01:36:12.232216+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-140"}]: dispatch 2024-02-18T01:36:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[19734]: audit 2024-02-18T01:36:12.189755+0000 mon.a (mon.0) 3181 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[19734]: cluster 2024-02-18T01:36:12.189905+0000 mon.a (mon.0) 3182 : cluster [DBG] osdmap e709: 8 total, 8 up, 8 in 2024-02-18T01:36:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[19734]: audit 2024-02-18T01:36:12.231828+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:13.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:13 smithi105 bash[19734]: audit 2024-02-18T01:36:12.232216+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-140"}]: dispatch 2024-02-18T01:36:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:13 smithi155 bash[20805]: audit 2024-02-18T01:36:12.231828+0000 mon.a (mon.0) 3183 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:13.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:13 smithi155 bash[20805]: audit 2024-02-18T01:36:12.232216+0000 mon.a (mon.0) 3184 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-140"}]: dispatch 2024-02-18T01:36:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[19734]: cluster 2024-02-18T01:36:13.205956+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T01:36:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[19734]: cluster 2024-02-18T01:36:14.027647+0000 mgr.y (mgr.24475) 738 : cluster [DBG] pgmap v1211: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:36:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[19734]: cluster 2024-02-18T01:36:14.212323+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T01:36:14.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[19734]: audit 2024-02-18T01:36:14.213876+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[27669]: cluster 2024-02-18T01:36:13.205956+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T01:36:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[27669]: cluster 2024-02-18T01:36:14.027647+0000 mgr.y (mgr.24475) 738 : cluster [DBG] pgmap v1211: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:36:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[27669]: cluster 2024-02-18T01:36:14.212323+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T01:36:14.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:14 smithi105 bash[27669]: audit 2024-02-18T01:36:14.213876+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:14 smithi155 bash[20805]: cluster 2024-02-18T01:36:13.205956+0000 mon.a (mon.0) 3185 : cluster [DBG] osdmap e710: 8 total, 8 up, 8 in 2024-02-18T01:36:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:14 smithi155 bash[20805]: cluster 2024-02-18T01:36:14.027647+0000 mgr.y (mgr.24475) 738 : cluster [DBG] pgmap v1211: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-02-18T01:36:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:14 smithi155 bash[20805]: cluster 2024-02-18T01:36:14.212323+0000 mon.a (mon.0) 3186 : cluster [DBG] osdmap e711: 8 total, 8 up, 8 in 2024-02-18T01:36:14.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:14 smithi155 bash[20805]: audit 2024-02-18T01:36:14.213876+0000 mon.a (mon.0) 3187 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[27669]: audit 2024-02-18T01:36:15.211051+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[27669]: cluster 2024-02-18T01:36:15.211408+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T01:36:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[27669]: audit 2024-02-18T01:36:15.267772+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[27669]: audit 2024-02-18T01:36:15.268256+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-142"}]: dispatch 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[27669]: cluster 2024-02-18T01:36:16.028518+0000 mgr.y (mgr.24475) 739 : cluster [DBG] pgmap v1214: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[19734]: audit 2024-02-18T01:36:15.211051+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[19734]: cluster 2024-02-18T01:36:15.211408+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[19734]: audit 2024-02-18T01:36:15.267772+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[19734]: audit 2024-02-18T01:36:15.268256+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-142"}]: dispatch 2024-02-18T01:36:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[19734]: cluster 2024-02-18T01:36:16.028518+0000 mgr.y (mgr.24475) 739 : cluster [DBG] pgmap v1214: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:36:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:16 smithi155 bash[20805]: audit 2024-02-18T01:36:15.211051+0000 mon.a (mon.0) 3188 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:16 smithi155 bash[20805]: cluster 2024-02-18T01:36:15.211408+0000 mon.a (mon.0) 3189 : cluster [DBG] osdmap e712: 8 total, 8 up, 8 in 2024-02-18T01:36:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:16 smithi155 bash[20805]: audit 2024-02-18T01:36:15.267772+0000 mon.a (mon.0) 3190 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:16 smithi155 bash[20805]: audit 2024-02-18T01:36:15.268256+0000 mon.a (mon.0) 3191 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-142"}]: dispatch 2024-02-18T01:36:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:16 smithi155 bash[20805]: cluster 2024-02-18T01:36:16.028518+0000 mgr.y (mgr.24475) 739 : cluster [DBG] pgmap v1214: 204 pgs: 32 unknown, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 2 op/s 2024-02-18T01:36:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:16] "GET /metrics HTTP/1.1" 200 191538 "" "Prometheus/2.33.4" 2024-02-18T01:36:17.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:17 smithi105 bash[19734]: cluster 2024-02-18T01:36:16.221124+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T01:36:17.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:17 smithi105 bash[27669]: cluster 2024-02-18T01:36:16.221124+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T01:36:17.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:17 smithi155 bash[20805]: cluster 2024-02-18T01:36:16.221124+0000 mon.a (mon.0) 3192 : cluster [DBG] osdmap e713: 8 total, 8 up, 8 in 2024-02-18T01:36:18.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[19734]: cluster 2024-02-18T01:36:17.232073+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T01:36:18.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[19734]: audit 2024-02-18T01:36:17.235325+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:18.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[19734]: cluster 2024-02-18T01:36:18.030494+0000 mgr.y (mgr.24475) 740 : cluster [DBG] pgmap v1217: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[27669]: cluster 2024-02-18T01:36:17.232073+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T01:36:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[27669]: audit 2024-02-18T01:36:17.235325+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:18.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:18 smithi105 bash[27669]: cluster 2024-02-18T01:36:18.030494+0000 mgr.y (mgr.24475) 740 : cluster [DBG] pgmap v1217: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:18.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:18 smithi155 bash[20805]: cluster 2024-02-18T01:36:17.232073+0000 mon.a (mon.0) 3193 : cluster [DBG] osdmap e714: 8 total, 8 up, 8 in 2024-02-18T01:36:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:18 smithi155 bash[20805]: audit 2024-02-18T01:36:17.235325+0000 mon.a (mon.0) 3194 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-02-18T01:36:18.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:18 smithi155 bash[20805]: cluster 2024-02-18T01:36:18.030494+0000 mgr.y (mgr.24475) 740 : cluster [DBG] pgmap v1217: 204 pgs: 32 creating+peering, 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:19.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: audit 2024-02-18T01:36:18.239261+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:19.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: cluster 2024-02-18T01:36:18.239780+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T01:36:19.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: audit 2024-02-18T01:36:18.251888+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-101","var": "dedup_tier","val": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: audit 2024-02-18T01:36:18.269470+0000 mgr.y (mgr.24475) 741 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: audit 2024-02-18T01:36:18.272374+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[19734]: audit 2024-02-18T01:36:18.272790+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: audit 2024-02-18T01:36:18.239261+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: cluster 2024-02-18T01:36:18.239780+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T01:36:19.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: audit 2024-02-18T01:36:18.251888+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-101","var": "dedup_tier","val": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: audit 2024-02-18T01:36:18.269470+0000 mgr.y (mgr.24475) 741 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: audit 2024-02-18T01:36:18.272374+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:19.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:19 smithi105 bash[27669]: audit 2024-02-18T01:36:18.272790+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: audit 2024-02-18T01:36:18.239261+0000 mon.a (mon.0) 3195 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi105-61833-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: cluster 2024-02-18T01:36:18.239780+0000 mon.a (mon.0) 3196 : cluster [DBG] osdmap e715: 8 total, 8 up, 8 in 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: audit 2024-02-18T01:36:18.251888+0000 mon.a (mon.0) 3197 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi105-61833-101","var": "dedup_tier","val": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: audit 2024-02-18T01:36:18.269470+0000 mgr.y (mgr.24475) 741 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: audit 2024-02-18T01:36:18.272374+0000 mon.a (mon.0) 3198 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:19.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:19 smithi155 bash[20805]: audit 2024-02-18T01:36:18.272790+0000 mon.a (mon.0) 3199 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi105-61833-101", "tierpool": "test-rados-api-smithi105-61833-144"}]: dispatch 2024-02-18T01:36:20.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[27669]: cluster 2024-02-18T01:36:19.255647+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T01:36:20.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[27669]: cluster 2024-02-18T01:36:19.646591+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:20.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[27669]: cluster 2024-02-18T01:36:20.031333+0000 mgr.y (mgr.24475) 742 : cluster [DBG] pgmap v1220: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:20.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[27669]: cluster 2024-02-18T01:36:20.262583+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T01:36:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[19734]: cluster 2024-02-18T01:36:19.255647+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T01:36:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[19734]: cluster 2024-02-18T01:36:19.646591+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[19734]: cluster 2024-02-18T01:36:20.031333+0000 mgr.y (mgr.24475) 742 : cluster [DBG] pgmap v1220: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:20.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:20 smithi105 bash[19734]: cluster 2024-02-18T01:36:20.262583+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T01:36:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:20 smithi155 bash[20805]: cluster 2024-02-18T01:36:19.255647+0000 mon.a (mon.0) 3200 : cluster [DBG] osdmap e716: 8 total, 8 up, 8 in 2024-02-18T01:36:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:20 smithi155 bash[20805]: cluster 2024-02-18T01:36:19.646591+0000 mon.a (mon.0) 3201 : cluster [WRN] Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:20 smithi155 bash[20805]: cluster 2024-02-18T01:36:20.031333+0000 mgr.y (mgr.24475) 742 : cluster [DBG] pgmap v1220: 172 pgs: 172 active+clean; 3.8 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-02-18T01:36:20.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:20 smithi155 bash[20805]: cluster 2024-02-18T01:36:20.262583+0000 mon.a (mon.0) 3202 : cluster [DBG] osdmap e717: 8 total, 8 up, 8 in 2024-02-18T01:36:21.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[19734]: audit 2024-02-18T01:36:20.265051+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[19734]: audit 2024-02-18T01:36:21.260408+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[19734]: cluster 2024-02-18T01:36:21.260486+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[19734]: audit 2024-02-18T01:36:21.263416+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[27669]: audit 2024-02-18T01:36:20.265051+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[27669]: audit 2024-02-18T01:36:21.260408+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:21.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[27669]: cluster 2024-02-18T01:36:21.260486+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T01:36:21.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:21 smithi105 bash[27669]: audit 2024-02-18T01:36:21.263416+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:21 smithi155 bash[20805]: audit 2024-02-18T01:36:20.265051+0000 mon.a (mon.0) 3203 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:21 smithi155 bash[20805]: audit 2024-02-18T01:36:21.260408+0000 mon.a (mon.0) 3204 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:21 smithi155 bash[20805]: cluster 2024-02-18T01:36:21.260486+0000 mon.a (mon.0) 3205 : cluster [DBG] osdmap e718: 8 total, 8 up, 8 in 2024-02-18T01:36:21.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:21 smithi155 bash[20805]: audit 2024-02-18T01:36:21.263416+0000 mon.a (mon.0) 3206 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]: dispatch 2024-02-18T01:36:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:22.271 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9068 ms) 2024-02-18T01:36:22.271 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: first is 1708220097 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,1708220105,1708220106,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,1708220105,1708220106,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220097,1708220099,1708220100,1708220102,1708220103,1708220105,1708220106,0 2024-02-18T01:36:22.272 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: got ls 1708220100,1708220102,1708220103,1708220105,1708220106,1708220108,1708220109,0 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: first now 1708220100, trimmed 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20441 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: foo0 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14113 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17917 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23018 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5050 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3023 ms) 2024-02-18T01:36:22.273 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3014 ms) 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3036 ms) 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231290 ms total) 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [==========] 72 tests from 4 test suites ran. (802234 ms total) 2024-02-18T01:36:22.274 INFO:tasks.workunit.client.0.smithi105.stdout: api_tier_pp: [ PASSED ] 72 tests. 2024-02-18T01:36:22.275 INFO:tasks.workunit.client.0.smithi105.stderr:+ exit 0 2024-02-18T01:36:22.275 INFO:tasks.workunit.client.0.smithi105.stderr:+ cleanup 2024-02-18T01:36:22.275 INFO:tasks.workunit.client.0.smithi105.stderr:+ pkill -P 61639 2024-02-18T01:36:22.293 INFO:tasks.workunit.client.0.smithi105.stderr:+ true 2024-02-18T01:36:22.294 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-18T01:36:22.294 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-18T01:36:22.304 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-02-18T01:36:22.305 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-02-18T01:36:22.306 DEBUG:teuthology.orchestra.run.smithi105:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=81bd20d634209c7cb82c18be12b4b5a05643ebf1 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-02-18T01:36:22.367 INFO:tasks.workunit.client.0.smithi105.stderr:+ uuidgen 2024-02-18T01:36:22.368 INFO:tasks.workunit.client.0.smithi105.stderr:+ p=6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:36:22.368 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool create 6cfab109-eaaa-440b-98e0-f8450a870fb2 12 2024-02-18T01:36:22.508 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.501+0000 7f138dce1700 1 -- 172.21.15.105:0/3181049987 >> v1:172.21.15.105:6789/0 conn(0x7f13880fc870 legacy=0x7f13880fcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:22.509 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.501+0000 7f138dce1700 1 -- 172.21.15.105:0/3181049987 shutdown_connections 2024-02-18T01:36:22.509 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.501+0000 7f138dce1700 1 -- 172.21.15.105:0/3181049987 >> 172.21.15.105:0/3181049987 conn(0x7f13880f24b0 msgr2=0x7f13880f48d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:22.509 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.501+0000 7f138dce1700 1 -- 172.21.15.105:0/3181049987 shutdown_connections 2024-02-18T01:36:22.509 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.501+0000 7f138dce1700 1 -- 172.21.15.105:0/3181049987 wait complete. 2024-02-18T01:36:22.509 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 Processor -- start 2024-02-18T01:36:22.510 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- start start 2024-02-18T01:36:22.512 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1388061ae0 con 0x7f13880fc870 2024-02-18T01:36:22.512 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f1388118650 con 0x7f13880ff690 2024-02-18T01:36:22.512 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f13881197f0 con 0x7f13881031a0 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1387fff700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f13881031a0 0x7f1388115f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:48498/0 (socket says 172.21.15.105:48498) 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f13877fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f13880fc870 0x7f1388060f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33068/0 (socket says 172.21.15.105:33068) 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1387fff700 1 -- 172.21.15.105:0/2316759687 learned_addr learned my addr 172.21.15.105:0/2316759687 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838194901 0 0) 0x7f13881197f0 con 0x7f13881031a0 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13700033b0 con 0x7f13881031a0 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2928936452 0 0) 0x7f1388061ae0 con 0x7f13880fc870 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13881197f0 con 0x7f13880fc870 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076662796 0 0) 0x7f1388118650 con 0x7f13880ff690 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1388061ae0 con 0x7f13880ff690 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1173432036 0 0) 0x7f13881197f0 con 0x7f13880fc870 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f1388118650 con 0x7f13880fc870 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f1378002d10 con 0x7f13880fc870 2024-02-18T01:36:22.513 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1010285693 0 0) 0x7f13700033b0 con 0x7f13881031a0 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13881197f0 con 0x7f13881031a0 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f1380002c90 con 0x7f13881031a0 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3598427521 0 0) 0x7f1388061ae0 con 0x7f13880ff690 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f13700033b0 con 0x7f13880ff690 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f137c001840 con 0x7f13880ff690 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 4028385763 0 0) 0x7f1388118650 con 0x7f13880fc870 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 >> v1:172.21.15.105:6790/0 conn(0x7f13881031a0 legacy=0x7f1388115f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 >> v1:172.21.15.155:6789/0 conn(0x7f13880ff690 legacy=0x7f13881127f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138811a9f0 con 0x7f13880fc870 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13881176a0 con 0x7f13880fc870 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f1388117af0 con 0x7f13880fc870 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f1378001a80 con 0x7f13880fc870 2024-02-18T01:36:22.514 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.505+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f13780036f0 con 0x7f13880fc870 2024-02-18T01:36:22.515 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.509+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1374004ee0 con 0x7f13880fc870 2024-02-18T01:36:22.515 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.509+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f1378003930 con 0x7f13880fc870 2024-02-18T01:36:22.515 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.509+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(719..719 src has 1..719) v4 ==== 7210+0+0 (unknown 2814834740 0 0) 0x7f1378078c50 con 0x7f13880fc870 2024-02-18T01:36:22.523 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.517+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f137804c4b0 con 0x7f13880fc870 2024-02-18T01:36:22.702 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:22.697+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12} v 0) v1 -- 0x7f1374005710 con 0x7f13880fc870 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[19734]: cluster 2024-02-18T01:36:22.032087+0000 mgr.y (mgr.24475) 743 : cluster [DBG] pgmap v1223: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[19734]: audit 2024-02-18T01:36:22.265890+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[19734]: cluster 2024-02-18T01:36:22.265986+0000 mon.a (mon.0) 3208 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[27669]: cluster 2024-02-18T01:36:22.032087+0000 mgr.y (mgr.24475) 743 : cluster [DBG] pgmap v1223: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[27669]: audit 2024-02-18T01:36:22.265890+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:22.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:22 smithi105 bash[27669]: cluster 2024-02-18T01:36:22.265986+0000 mon.a (mon.0) 3208 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T01:36:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:22 smithi155 bash[20805]: cluster 2024-02-18T01:36:22.032087+0000 mgr.y (mgr.24475) 743 : cluster [DBG] pgmap v1223: 164 pgs: 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:22 smithi155 bash[20805]: audit 2024-02-18T01:36:22.265890+0000 mon.a (mon.0) 3207 : audit [INF] from='client.? v1:172.21.15.105:0/4162885829' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi105-61833-101"}]': finished 2024-02-18T01:36:22.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:22 smithi155 bash[20805]: cluster 2024-02-18T01:36:22.265986+0000 mon.a (mon.0) 3208 : cluster [DBG] osdmap e719: 8 total, 8 up, 8 in 2024-02-18T01:36:23.294 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.289+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]=0 pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' created v720) v1 ==== 176+0+0 (unknown 2737319695 0 0) 0x7f137804a920 con 0x7f13880fc870 2024-02-18T01:36:23.413 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12} v 0) v1 -- 0x7f13740058d0 con 0x7f13880fc870 2024-02-18T01:36:23.414 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f1384ff9700 1 -- 172.21.15.105:0/2316759687 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]=0 pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' already exists v720) v1 ==== 183+0+0 (unknown 777495561 0 0) 0x7f137804c0a0 con 0x7f13880fc870 2024-02-18T01:36:23.416 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 >> v1:172.21.15.105:6800/2148176012 conn(0x7f1370062280 legacy=0x7f1370064740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:23.416 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 >> v1:172.21.15.105:6789/0 conn(0x7f13880fc870 legacy=0x7f1388060f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:23.416 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 shutdown_connections 2024-02-18T01:36:23.417 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 >> 172.21.15.105:0/2316759687 conn(0x7f13880f24b0 msgr2=0x7f1388106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:23.417 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 shutdown_connections 2024-02-18T01:36:23.417 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.409+0000 7f138dce1700 1 -- 172.21.15.105:0/2316759687 wait complete. 2024-02-18T01:36:23.418 INFO:tasks.workunit.client.0.smithi105.stderr:pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' already exists 2024-02-18T01:36:23.428 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_objects 10 2024-02-18T01:36:23.554 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- 172.21.15.105:0/256903823 >> v1:172.21.15.105:6790/0 conn(0x7f35ec0f8940 legacy=0x7f35ec0f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:23.555 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- 172.21.15.105:0/256903823 shutdown_connections 2024-02-18T01:36:23.555 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- 172.21.15.105:0/256903823 >> 172.21.15.105:0/256903823 conn(0x7f35ec0f24d0 msgr2=0x7f35ec0f48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:23.555 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- 172.21.15.105:0/256903823 shutdown_connections 2024-02-18T01:36:23.555 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- 172.21.15.105:0/256903823 wait complete. 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 Processor -- start 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- start start 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f35ec119430 con 0x7f35ec0f9430 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f35ec11a5d0 con 0x7f35ec1042a0 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35f2938700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f35ec11b7d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35ebfff700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f35ec0f8940 0x7f35ec1021f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:48530/0 (socket says 172.21.15.105:48530) 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35eb7fe700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f35ec0f9430 0x7f35ec102900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33090/0 (socket says 172.21.15.105:33090) 2024-02-18T01:36:23.556 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35ebfff700 1 -- 172.21.15.105:0/419715943 learned_addr learned my addr 172.21.15.105:0/419715943 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:36:23.558 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3218279998 0 0) 0x7f35ec11b7d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.558 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35d8003110 con 0x7f35ec0f8940 2024-02-18T01:36:23.558 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254960346 0 0) 0x7f35ec119430 con 0x7f35ec0f9430 2024-02-18T01:36:23.558 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ec11b7d0 con 0x7f35ec0f9430 2024-02-18T01:36:23.558 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3721743852 0 0) 0x7f35ec11a5d0 con 0x7f35ec1042a0 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35ec119430 con 0x7f35ec1042a0 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2179207392 0 0) 0x7f35d8003110 con 0x7f35ec0f8940 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35ec11a5d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.549+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f35d4001590 con 0x7f35ec0f8940 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3752112263 0 0) 0x7f35ec11b7d0 con 0x7f35ec0f9430 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35d8003110 con 0x7f35ec0f9430 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f35dc001840 con 0x7f35ec0f9430 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2712892619 0 0) 0x7f35ec119430 con 0x7f35ec1042a0 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f35ec11b7d0 con 0x7f35ec1042a0 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f35e40019e0 con 0x7f35ec1042a0 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 367497284 0 0) 0x7f35ec11a5d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 >> v1:172.21.15.155:6789/0 conn(0x7f35ec1042a0 legacy=0x7f35ec117d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 >> v1:172.21.15.105:6789/0 conn(0x7f35ec0f9430 legacy=0x7f35ec102900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:23.559 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ec11c9d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.560 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ec119620 con 0x7f35ec0f8940 2024-02-18T01:36:23.560 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ec119a70 con 0x7f35ec0f8940 2024-02-18T01:36:23.560 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f35d40023b0 con 0x7f35ec0f8940 2024-02-18T01:36:23.560 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f35d4003870 con 0x7f35ec0f8940 2024-02-18T01:36:23.561 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f35d40039d0 con 0x7f35ec0f8940 2024-02-18T01:36:23.561 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35cc004ee0 con 0x7f35ec0f8940 2024-02-18T01:36:23.561 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.553+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(720..720 src has 1..720) v4 ==== 7585+0+0 (unknown 3753578258 0 0) 0x7f35d4079de0 con 0x7f35ec0f8940 2024-02-18T01:36:23.566 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.561+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f35d404d470 con 0x7f35ec0f8940 2024-02-18T01:36:23.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:23 smithi105 bash[27669]: audit 2024-02-18T01:36:22.702699+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:23.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:23 smithi105 bash[19734]: audit 2024-02-18T01:36:22.702699+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:23.742 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:23 smithi155 bash[20805]: audit 2024-02-18T01:36:22.702699+0000 mon.a (mon.0) 3209 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:23.744 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:23.737+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f35cc005190 con 0x7f35ec0f8940 2024-02-18T01:36:24.302 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:24.297+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v721) v1 ==== 223+0+0 (unknown 813973870 0 0) 0x7f35d404b8e0 con 0x7f35ec0f8940 2024-02-18T01:36:24.424 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:24.417+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f35cc002220 con 0x7f35ec0f8940 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: audit 2024-02-18T01:36:23.293930+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]': finished 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: cluster 2024-02-18T01:36:23.294026+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: audit 2024-02-18T01:36:23.413942+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: audit 2024-02-18T01:36:23.744559+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: audit 2024-02-18T01:36:23.745357+0000 mon.a (mon.0) 3213 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[19734]: cluster 2024-02-18T01:36:24.032927+0000 mgr.y (mgr.24475) 744 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: audit 2024-02-18T01:36:23.293930+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]': finished 2024-02-18T01:36:24.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: cluster 2024-02-18T01:36:23.294026+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T01:36:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: audit 2024-02-18T01:36:23.413942+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: audit 2024-02-18T01:36:23.744559+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: audit 2024-02-18T01:36:23.745357+0000 mon.a (mon.0) 3213 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.741 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:24 smithi105 bash[27669]: cluster 2024-02-18T01:36:24.032927+0000 mgr.y (mgr.24475) 744 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: audit 2024-02-18T01:36:23.293930+0000 mon.a (mon.0) 3210 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]': finished 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: cluster 2024-02-18T01:36:23.294026+0000 mon.a (mon.0) 3211 : cluster [DBG] osdmap e720: 8 total, 8 up, 8 in 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: audit 2024-02-18T01:36:23.413942+0000 mon.a (mon.0) 3212 : audit [INF] from='client.? v1:172.21.15.105:0/2316759687' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pg_num": 12}]: dispatch 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: audit 2024-02-18T01:36:23.744559+0000 mon.c (mon.2) 415 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: audit 2024-02-18T01:36:23.745357+0000 mon.a (mon.0) 3213 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:24.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:24 smithi155 bash[20805]: cluster 2024-02-18T01:36:24.032927+0000 mgr.y (mgr.24475) 744 : cluster [DBG] pgmap v1226: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:36:25.316 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.309+0000 7f35e97fa700 1 -- 172.21.15.105:0/419715943 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v722) v1 ==== 223+0+0 (unknown 3947540222 0 0) 0x7f35d404d060 con 0x7f35ec0f8940 2024-02-18T01:36:25.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.313+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 >> v1:172.21.15.105:6800/2148176012 conn(0x7f35d80622d0 legacy=0x7f35d8064790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:25.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.313+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 >> v1:172.21.15.105:6790/0 conn(0x7f35ec0f8940 legacy=0x7f35ec1021f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:25.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.313+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 shutdown_connections 2024-02-18T01:36:25.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.313+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 >> 172.21.15.105:0/419715943 conn(0x7f35ec0f24d0 msgr2=0x7f35ec107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:25.321 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.317+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 shutdown_connections 2024-02-18T01:36:25.321 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.317+0000 7f35f2938700 1 -- 172.21.15.105:0/419715943 wait complete. 2024-02-18T01:36:25.321 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_objects = 10 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:36:25.332 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool application enable 6cfab109-eaaa-440b-98e0-f8450a870fb2 rados 2024-02-18T01:36:25.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/3043279111 >> v1:172.21.15.105:6789/0 conn(0x7f09fc1064b0 legacy=0x7f09fc108920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:25.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/3043279111 shutdown_connections 2024-02-18T01:36:25.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/3043279111 >> 172.21.15.105:0/3043279111 conn(0x7f09fc0f57c0 msgr2=0x7f09fc0f7be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:25.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/3043279111 shutdown_connections 2024-02-18T01:36:25.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/3043279111 wait complete. 2024-02-18T01:36:25.457 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 Processor -- start 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- start start 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09fc119380 con 0x7f09fc1064b0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09fc11a560 con 0x7f09fc0ffb30 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f0a01e96700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09fc11b760 con 0x7f09fc1029a0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09fbfff700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f09fc1064b0 0x7f09fc117b00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:33106/0 (socket says 172.21.15.105:33106) 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09fbfff700 1 -- 172.21.15.105:0/1900331358 learned_addr learned my addr 172.21.15.105:0/1900331358 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09faffd700 1 --1- 172.21.15.105:0/1900331358 >> v1:172.21.15.105:6790/0 conn(0x7f09fc1029a0 0x7f09fc0fc130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:48546/0 (socket says 172.21.15.105:48546) 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 353529169 0 0) 0x7f09fc11b760 con 0x7f09fc1029a0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e80033b0 con 0x7f09fc1029a0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475623336 0 0) 0x7f09fc119380 con 0x7f09fc1064b0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc11b760 con 0x7f09fc1064b0 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3333500948 0 0) 0x7f09fc11a560 con 0x7f09fc0ffb30 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09fc119380 con 0x7f09fc0ffb30 2024-02-18T01:36:25.458 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2017842607 0 0) 0x7f09e80033b0 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09fc11a560 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.449+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09e4001990 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4143514082 0 0) 0x7f09fc11b760 con 0x7f09fc1064b0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09e80033b0 con 0x7f09fc1064b0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09f4001320 con 0x7f09fc1064b0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1844151796 0 0) 0x7f09fc11a560 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 >> v1:172.21.15.155:6789/0 conn(0x7f09fc0ffb30 legacy=0x7f09fc0fba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 >> v1:172.21.15.105:6789/0 conn(0x7f09fc1064b0 legacy=0x7f09fc117b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09fc11c960 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc11b950 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f09fc11be20 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f09e40023b0 con 0x7f09fc1029a0 2024-02-18T01:36:25.459 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09e4005a20 con 0x7f09fc1029a0 2024-02-18T01:36:25.460 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09dc004ee0 con 0x7f09fc1029a0 2024-02-18T01:36:25.460 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.453+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f09e4001f10 con 0x7f09fc1029a0 2024-02-18T01:36:25.460 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.457+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(722..722 src has 1..722) v4 ==== 7585+0+0 (unknown 3389191613 0 0) 0x7f09e4078ae0 con 0x7f09fc1029a0 2024-02-18T01:36:25.467 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.461+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f09e404c110 con 0x7f09fc1029a0 2024-02-18T01:36:25.641 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:25.637+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"} v 0) v1 -- 0x7f09dc005190 con 0x7f09fc1029a0 2024-02-18T01:36:25.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[19734]: audit 2024-02-18T01:36:24.298940+0000 mon.a (mon.0) 3214 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:25.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[19734]: cluster 2024-02-18T01:36:24.299130+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T01:36:25.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[19734]: audit 2024-02-18T01:36:24.425347+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[19734]: audit 2024-02-18T01:36:24.426054+0000 mon.a (mon.0) 3216 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[19734]: cluster 2024-02-18T01:36:24.648526+0000 mon.a (mon.0) 3217 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[27669]: audit 2024-02-18T01:36:24.298940+0000 mon.a (mon.0) 3214 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[27669]: cluster 2024-02-18T01:36:24.299130+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T01:36:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[27669]: audit 2024-02-18T01:36:24.425347+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[27669]: audit 2024-02-18T01:36:24.426054+0000 mon.a (mon.0) 3216 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:25 smithi105 bash[27669]: cluster 2024-02-18T01:36:24.648526+0000 mon.a (mon.0) 3217 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:25 smithi155 bash[20805]: audit 2024-02-18T01:36:24.298940+0000 mon.a (mon.0) 3214 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:25 smithi155 bash[20805]: cluster 2024-02-18T01:36:24.299130+0000 mon.a (mon.0) 3215 : cluster [DBG] osdmap e721: 8 total, 8 up, 8 in 2024-02-18T01:36:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:25 smithi155 bash[20805]: audit 2024-02-18T01:36:24.425347+0000 mon.c (mon.2) 416 : audit [INF] from='client.? v1:172.21.15.105:0/419715943' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:25 smithi155 bash[20805]: audit 2024-02-18T01:36:24.426054+0000 mon.a (mon.0) 3216 : audit [INF] from='client.48275 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:36:25.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:25 smithi155 bash[20805]: cluster 2024-02-18T01:36:24.648526+0000 mon.a (mon.0) 3217 : cluster [WRN] Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-02-18T01:36:26.338 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:26.333+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]=0 enabled application 'rados' on pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' v723) v1 ==== 213+0+0 (unknown 4188666358 0 0) 0x7f09e404a520 con 0x7f09fc1029a0 2024-02-18T01:36:26.455 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:26.449+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"} v 0) v1 -- 0x7f09dc002220 con 0x7f09fc1029a0 2024-02-18T01:36:26.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[27669]: audit 2024-02-18T01:36:25.314775+0000 mon.a (mon.0) 3218 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:26.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[27669]: cluster 2024-02-18T01:36:25.314968+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T01:36:26.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[27669]: audit 2024-02-18T01:36:25.641712+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[27669]: audit 2024-02-18T01:36:25.642213+0000 mon.a (mon.0) 3220 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[27669]: cluster 2024-02-18T01:36:26.033775+0000 mgr.y (mgr.24475) 745 : cluster [DBG] pgmap v1229: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[19734]: audit 2024-02-18T01:36:25.314775+0000 mon.a (mon.0) 3218 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[19734]: cluster 2024-02-18T01:36:25.314968+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T01:36:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[19734]: audit 2024-02-18T01:36:25.641712+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[19734]: audit 2024-02-18T01:36:25.642213+0000 mon.a (mon.0) 3220 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.679 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[19734]: cluster 2024-02-18T01:36:26.033775+0000 mgr.y (mgr.24475) 745 : cluster [DBG] pgmap v1229: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:26 smithi155 bash[20805]: audit 2024-02-18T01:36:25.314775+0000 mon.a (mon.0) 3218 : audit [INF] from='client.48275 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:36:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:26 smithi155 bash[20805]: cluster 2024-02-18T01:36:25.314968+0000 mon.a (mon.0) 3219 : cluster [DBG] osdmap e722: 8 total, 8 up, 8 in 2024-02-18T01:36:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:26 smithi155 bash[20805]: audit 2024-02-18T01:36:25.641712+0000 mon.c (mon.2) 417 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:26 smithi155 bash[20805]: audit 2024-02-18T01:36:25.642213+0000 mon.a (mon.0) 3220 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:26.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:26 smithi155 bash[20805]: cluster 2024-02-18T01:36:26.033775+0000 mgr.y (mgr.24475) 745 : cluster [DBG] pgmap v1229: 176 pgs: 12 unknown, 164 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:26] "GET /metrics HTTP/1.1" 200 186654 "" "Prometheus/2.33.4" 2024-02-18T01:36:27.348 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.341+0000 7f09f8ff9700 1 -- 172.21.15.105:0/1900331358 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]=0 enabled application 'rados' on pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' v724) v1 ==== 213+0+0 (unknown 1598830916 0 0) 0x7f09e404bca0 con 0x7f09fc1029a0 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 >> v1:172.21.15.105:6800/2148176012 conn(0x7f09e8062240 legacy=0x7f09e8064700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 >> v1:172.21.15.105:6790/0 conn(0x7f09fc1029a0 legacy=0x7f09fc0fc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 shutdown_connections 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 >> 172.21.15.105:0/1900331358 conn(0x7f09fc0f57c0 msgr2=0x7f09fc109910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 shutdown_connections 2024-02-18T01:36:27.351 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:27.345+0000 7f0a01e96700 1 -- 172.21.15.105:0/1900331358 wait complete. 2024-02-18T01:36:27.352 INFO:tasks.workunit.client.0.smithi105.stderr:enabled application 'rados' on pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' 2024-02-18T01:36:27.364 INFO:tasks.workunit.client.0.smithi105.stderr:+ seq 1 10 2024-02-18T01:36:27.383 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj1 /etc/passwd 2024-02-18T01:36:27.431 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj2 /etc/passwd 2024-02-18T01:36:27.472 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj3 /etc/passwd 2024-02-18T01:36:27.513 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj4 /etc/passwd 2024-02-18T01:36:27.557 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj5 /etc/passwd 2024-02-18T01:36:27.599 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj6 /etc/passwd 2024-02-18T01:36:27.639 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj7 /etc/passwd 2024-02-18T01:36:27.680 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj8 /etc/passwd 2024-02-18T01:36:27.722 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj9 /etc/passwd 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[27669]: audit 2024-02-18T01:36:26.335458+0000 mon.a (mon.0) 3221 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[27669]: cluster 2024-02-18T01:36:26.335662+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[27669]: audit 2024-02-18T01:36:26.455436+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[27669]: audit 2024-02-18T01:36:26.456002+0000 mon.a (mon.0) 3223 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[19734]: audit 2024-02-18T01:36:26.335458+0000 mon.a (mon.0) 3221 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[19734]: cluster 2024-02-18T01:36:26.335662+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T01:36:27.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[19734]: audit 2024-02-18T01:36:26.455436+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:27 smithi105 bash[19734]: audit 2024-02-18T01:36:26.456002+0000 mon.a (mon.0) 3223 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:27 smithi155 bash[20805]: audit 2024-02-18T01:36:26.335458+0000 mon.a (mon.0) 3221 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:27 smithi155 bash[20805]: cluster 2024-02-18T01:36:26.335662+0000 mon.a (mon.0) 3222 : cluster [DBG] osdmap e723: 8 total, 8 up, 8 in 2024-02-18T01:36:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:27 smithi155 bash[20805]: audit 2024-02-18T01:36:26.455436+0000 mon.c (mon.2) 418 : audit [INF] from='client.? v1:172.21.15.105:0/1900331358' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:27 smithi155 bash[20805]: audit 2024-02-18T01:36:26.456002+0000 mon.a (mon.0) 3223 : audit [INF] from='client.48281 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]: dispatch 2024-02-18T01:36:27.763 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put obj10 /etc/passwd 2024-02-18T01:36:27.804 INFO:tasks.workunit.client.0.smithi105.stderr:+ sleep 30 2024-02-18T01:36:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: audit 2024-02-18T01:36:27.348004+0000 mon.a (mon.0) 3224 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: cluster 2024-02-18T01:36:27.348255+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T01:36:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: audit 2024-02-18T01:36:27.770749+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:36:28.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: cluster 2024-02-18T01:36:28.034913+0000 mgr.y (mgr.24475) 746 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: audit 2024-02-18T01:36:28.066752+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: audit 2024-02-18T01:36:28.067999+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[27669]: audit 2024-02-18T01:36:28.073938+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: audit 2024-02-18T01:36:27.348004+0000 mon.a (mon.0) 3224 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: cluster 2024-02-18T01:36:27.348255+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: audit 2024-02-18T01:36:27.770749+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: cluster 2024-02-18T01:36:28.034913+0000 mgr.y (mgr.24475) 746 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: audit 2024-02-18T01:36:28.066752+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: audit 2024-02-18T01:36:28.067999+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:36:28.740 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:28 smithi105 bash[19734]: audit 2024-02-18T01:36:28.073938+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:28.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: audit 2024-02-18T01:36:27.348004+0000 mon.a (mon.0) 3224 : audit [INF] from='client.48281 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "app": "rados"}]': finished 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: cluster 2024-02-18T01:36:27.348255+0000 mon.a (mon.0) 3225 : cluster [DBG] osdmap e724: 8 total, 8 up, 8 in 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: audit 2024-02-18T01:36:27.770749+0000 mon.a (mon.0) 3226 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: cluster 2024-02-18T01:36:28.034913+0000 mgr.y (mgr.24475) 746 : cluster [DBG] pgmap v1232: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: audit 2024-02-18T01:36:28.066752+0000 mon.a (mon.0) 3227 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: audit 2024-02-18T01:36:28.067999+0000 mon.a (mon.0) 3228 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:36:28.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:28 smithi155 bash[20805]: audit 2024-02-18T01:36:28.073938+0000 mon.a (mon.0) 3229 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:29.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:29 smithi105 bash[19734]: audit 2024-02-18T01:36:28.280484+0000 mgr.y (mgr.24475) 747 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:29.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:29 smithi105 bash[27669]: audit 2024-02-18T01:36:28.280484+0000 mgr.y (mgr.24475) 747 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:29.742 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:29 smithi155 bash[20805]: audit 2024-02-18T01:36:28.280484+0000 mgr.y (mgr.24475) 747 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:30.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:30 smithi105 bash[19734]: cluster 2024-02-18T01:36:30.035689+0000 mgr.y (mgr.24475) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-18T01:36:30.739 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:30 smithi105 bash[27669]: cluster 2024-02-18T01:36:30.035689+0000 mgr.y (mgr.24475) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-18T01:36:30.742 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:30 smithi155 bash[20805]: cluster 2024-02-18T01:36:30.035689+0000 mgr.y (mgr.24475) 748 : cluster [DBG] pgmap v1233: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 891 B/s rd, 0 op/s 2024-02-18T01:36:32.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:33.239 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:32 smithi105 bash[27669]: cluster 2024-02-18T01:36:32.036498+0000 mgr.y (mgr.24475) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 761 B/s rd, 0 op/s 2024-02-18T01:36:33.240 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:32 smithi105 bash[19734]: cluster 2024-02-18T01:36:32.036498+0000 mgr.y (mgr.24475) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 761 B/s rd, 0 op/s 2024-02-18T01:36:33.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:32 smithi155 bash[20805]: cluster 2024-02-18T01:36:32.036498+0000 mgr.y (mgr.24475) 749 : cluster [DBG] pgmap v1234: 176 pgs: 176 active+clean; 5.7 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 761 B/s rd, 0 op/s 2024-02-18T01:36:35.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[27669]: cluster 2024-02-18T01:36:34.038507+0000 mgr.y (mgr.24475) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-18T01:36:35.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[27669]: cluster 2024-02-18T01:36:34.651091+0000 mon.a (mon.0) 3230 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_objects: 10) 2024-02-18T01:36:35.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[27669]: cluster 2024-02-18T01:36:34.651585+0000 mon.a (mon.0) 3231 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:36:35.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[27669]: cluster 2024-02-18T01:36:34.663094+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T01:36:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[19734]: cluster 2024-02-18T01:36:34.038507+0000 mgr.y (mgr.24475) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-18T01:36:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[19734]: cluster 2024-02-18T01:36:34.651091+0000 mon.a (mon.0) 3230 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_objects: 10) 2024-02-18T01:36:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[19734]: cluster 2024-02-18T01:36:34.651585+0000 mon.a (mon.0) 3231 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:36:35.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:35 smithi105 bash[19734]: cluster 2024-02-18T01:36:34.663094+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T01:36:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:35 smithi155 bash[20805]: cluster 2024-02-18T01:36:34.038507+0000 mgr.y (mgr.24475) 750 : cluster [DBG] pgmap v1235: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-02-18T01:36:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:35 smithi155 bash[20805]: cluster 2024-02-18T01:36:34.651091+0000 mon.a (mon.0) 3230 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_objects: 10) 2024-02-18T01:36:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:35 smithi155 bash[20805]: cluster 2024-02-18T01:36:34.651585+0000 mon.a (mon.0) 3231 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:36:35.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:35 smithi155 bash[20805]: cluster 2024-02-18T01:36:34.663094+0000 mon.a (mon.0) 3232 : cluster [DBG] osdmap e725: 8 total, 8 up, 8 in 2024-02-18T01:36:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:36] "GET /metrics HTTP/1.1" 200 189309 "" "Prometheus/2.33.4" 2024-02-18T01:36:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:37 smithi105 bash[19734]: cluster 2024-02-18T01:36:36.039295+0000 mgr.y (mgr.24475) 751 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-18T01:36:37.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:37 smithi105 bash[19734]: audit 2024-02-18T01:36:36.524544+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:37.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:37 smithi105 bash[27669]: cluster 2024-02-18T01:36:36.039295+0000 mgr.y (mgr.24475) 751 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-18T01:36:37.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:37 smithi105 bash[27669]: audit 2024-02-18T01:36:36.524544+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:37 smithi155 bash[20805]: cluster 2024-02-18T01:36:36.039295+0000 mgr.y (mgr.24475) 751 : cluster [DBG] pgmap v1237: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-02-18T01:36:37.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:37 smithi155 bash[20805]: audit 2024-02-18T01:36:36.524544+0000 mon.a (mon.0) 3233 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:36:38.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:39.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:39 smithi105 bash[27669]: cluster 2024-02-18T01:36:38.041159+0000 mgr.y (mgr.24475) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:39 smithi105 bash[19734]: cluster 2024-02-18T01:36:38.041159+0000 mgr.y (mgr.24475) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:39 smithi155 bash[20805]: cluster 2024-02-18T01:36:38.041159+0000 mgr.y (mgr.24475) 752 : cluster [DBG] pgmap v1238: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:40.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:40 smithi105 bash[27669]: audit 2024-02-18T01:36:38.288776+0000 mgr.y (mgr.24475) 753 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:40.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:40 smithi105 bash[19734]: audit 2024-02-18T01:36:38.288776+0000 mgr.y (mgr.24475) 753 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:40 smithi155 bash[20805]: audit 2024-02-18T01:36:38.288776+0000 mgr.y (mgr.24475) 753 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:41.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:41 smithi105 bash[27669]: cluster 2024-02-18T01:36:40.041986+0000 mgr.y (mgr.24475) 754 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:41 smithi105 bash[19734]: cluster 2024-02-18T01:36:40.041986+0000 mgr.y (mgr.24475) 754 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:41 smithi155 bash[20805]: cluster 2024-02-18T01:36:40.041986+0000 mgr.y (mgr.24475) 754 : cluster [DBG] pgmap v1239: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:43.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:43 smithi105 bash[19734]: cluster 2024-02-18T01:36:42.042779+0000 mgr.y (mgr.24475) 755 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:43.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:43 smithi105 bash[27669]: cluster 2024-02-18T01:36:42.042779+0000 mgr.y (mgr.24475) 755 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:43.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:43 smithi155 bash[20805]: cluster 2024-02-18T01:36:42.042779+0000 mgr.y (mgr.24475) 755 : cluster [DBG] pgmap v1240: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:36:45.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:45 smithi105 bash[27669]: cluster 2024-02-18T01:36:44.044750+0000 mgr.y (mgr.24475) 756 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:36:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:45 smithi105 bash[19734]: cluster 2024-02-18T01:36:44.044750+0000 mgr.y (mgr.24475) 756 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:36:45.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:45 smithi155 bash[20805]: cluster 2024-02-18T01:36:44.044750+0000 mgr.y (mgr.24475) 756 : cluster [DBG] pgmap v1241: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:36:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:46] "GET /metrics HTTP/1.1" 200 189309 "" "Prometheus/2.33.4" 2024-02-18T01:36:47.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:47 smithi105 bash[27669]: cluster 2024-02-18T01:36:46.045835+0000 mgr.y (mgr.24475) 757 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 0 op/s 2024-02-18T01:36:47.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:47 smithi105 bash[19734]: cluster 2024-02-18T01:36:46.045835+0000 mgr.y (mgr.24475) 757 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 0 op/s 2024-02-18T01:36:47.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:47 smithi155 bash[20805]: cluster 2024-02-18T01:36:46.045835+0000 mgr.y (mgr.24475) 757 : cluster [DBG] pgmap v1242: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 899 B/s rd, 0 op/s 2024-02-18T01:36:48.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:49 smithi105 bash[19734]: cluster 2024-02-18T01:36:48.047680+0000 mgr.y (mgr.24475) 758 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:49.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:49 smithi105 bash[27669]: cluster 2024-02-18T01:36:48.047680+0000 mgr.y (mgr.24475) 758 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:49 smithi155 bash[20805]: cluster 2024-02-18T01:36:48.047680+0000 mgr.y (mgr.24475) 758 : cluster [DBG] pgmap v1243: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:50 smithi155 bash[20805]: audit 2024-02-18T01:36:48.296694+0000 mgr.y (mgr.24475) 759 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:50.512 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:50 smithi105 bash[27669]: audit 2024-02-18T01:36:48.296694+0000 mgr.y (mgr.24475) 759 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:50.513 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:50 smithi105 bash[19734]: audit 2024-02-18T01:36:48.296694+0000 mgr.y (mgr.24475) 759 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:36:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:51 smithi105 bash[19734]: cluster 2024-02-18T01:36:50.048477+0000 mgr.y (mgr.24475) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:51.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:51 smithi105 bash[27669]: cluster 2024-02-18T01:36:50.048477+0000 mgr.y (mgr.24475) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:51 smithi155 bash[20805]: cluster 2024-02-18T01:36:50.048477+0000 mgr.y (mgr.24475) 760 : cluster [DBG] pgmap v1244: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:52.162 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:36:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:36:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:52 smithi105 bash[27669]: cluster 2024-02-18T01:36:52.049286+0000 mgr.y (mgr.24475) 761 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:52.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:52 smithi105 bash[19734]: cluster 2024-02-18T01:36:52.049286+0000 mgr.y (mgr.24475) 761 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:52.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:52 smithi155 bash[20805]: cluster 2024-02-18T01:36:52.049286+0000 mgr.y (mgr.24475) 761 : cluster [DBG] pgmap v1245: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:55.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:55 smithi105 bash[27669]: cluster 2024-02-18T01:36:54.051142+0000 mgr.y (mgr.24475) 762 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:55.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:55 smithi105 bash[19734]: cluster 2024-02-18T01:36:54.051142+0000 mgr.y (mgr.24475) 762 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:55 smithi155 bash[20805]: cluster 2024-02-18T01:36:54.051142+0000 mgr.y (mgr.24475) 762 : cluster [DBG] pgmap v1246: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:36:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:36:56] "GET /metrics HTTP/1.1" 200 189307 "" "Prometheus/2.33.4" 2024-02-18T01:36:57.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:57 smithi105 bash[27669]: cluster 2024-02-18T01:36:56.051928+0000 mgr.y (mgr.24475) 763 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:57.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:57 smithi105 bash[19734]: cluster 2024-02-18T01:36:56.051928+0000 mgr.y (mgr.24475) 763 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:57.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:57 smithi155 bash[20805]: cluster 2024-02-18T01:36:56.051928+0000 mgr.y (mgr.24475) 763 : cluster [DBG] pgmap v1247: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:36:57.805 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=128407 2024-02-18T01:36:57.805 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_objects 100 2024-02-18T01:36:57.806 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put onemore /etc/passwd 2024-02-18T01:36:57.935 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- 172.21.15.105:0/333346486 >> v1:172.21.15.105:6789/0 conn(0x7f09f40f9430 legacy=0x7f09f41029c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:57.935 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- 172.21.15.105:0/333346486 shutdown_connections 2024-02-18T01:36:57.935 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- 172.21.15.105:0/333346486 >> 172.21.15.105:0/333346486 conn(0x7f09f40f24d0 msgr2=0x7f09f40f48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:57.935 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- 172.21.15.105:0/333346486 shutdown_connections 2024-02-18T01:36:57.936 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- 172.21.15.105:0/333346486 wait complete. 2024-02-18T01:36:57.939 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 Processor -- start 2024-02-18T01:36:57.939 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- start start 2024-02-18T01:36:57.939 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09f4119430 con 0x7f09f41042a0 2024-02-18T01:36:57.939 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09f411a5d0 con 0x7f09f40f8940 2024-02-18T01:36:57.939 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fbf89700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f09f411b770 con 0x7f09f40f9430 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09f9524700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f09f40f9430 0x7f09f4102860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:57624/0 (socket says 172.21.15.105:57624) 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09f9524700 1 -- 172.21.15.105:0/819780126 learned_addr learned my addr 172.21.15.105:0/819780126 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09fa526700 1 --1- 172.21.15.105:0/819780126 >> v1:172.21.15.105:6789/0 conn(0x7f09f41042a0 0x7f09f4117d10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:52032/0 (socket says 172.21.15.105:52032) 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 532867309 0 0) 0x7f09f4119430 con 0x7f09f41042a0 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09e00033b0 con 0x7f09f41042a0 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791817233 0 0) 0x7f09f411b770 con 0x7f09f40f9430 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f4119430 con 0x7f09f40f9430 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1211822042 0 0) 0x7f09f411a5d0 con 0x7f09f40f8940 2024-02-18T01:36:57.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.932+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f411b770 con 0x7f09f40f8940 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 224978894 0 0) 0x7f09e00033b0 con 0x7f09f41042a0 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09f411a5d0 con 0x7f09f41042a0 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09f00018a0 con 0x7f09f41042a0 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2358480711 0 0) 0x7f09f4119430 con 0x7f09f40f9430 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09e00033b0 con 0x7f09f40f9430 2024-02-18T01:36:57.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09e4002ca0 con 0x7f09f40f9430 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2684742570 0 0) 0x7f09f411b770 con 0x7f09f40f8940 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f09f4119430 con 0x7f09f40f8940 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09ec001c00 con 0x7f09f40f8940 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 3740821701 0 0) 0x7f09f411a5d0 con 0x7f09f41042a0 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 >> v1:172.21.15.105:6790/0 conn(0x7f09f40f9430 legacy=0x7f09f4102860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 >> v1:172.21.15.155:6789/0 conn(0x7f09f40f8940 legacy=0x7f09f4102150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f411c950 con 0x7f09f41042a0 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f4119620 con 0x7f09f41042a0 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f4119a70 con 0x7f09f41042a0 2024-02-18T01:36:57.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f09f0004ba0 con 0x7f09f41042a0 2024-02-18T01:36:57.943 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f09f0005a10 con 0x7f09f41042a0 2024-02-18T01:36:57.943 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09d8004ee0 con 0x7f09f41042a0 2024-02-18T01:36:57.943 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f09f0018980 con 0x7f09f41042a0 2024-02-18T01:36:57.944 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.936+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(725..725 src has 1..725) v4 ==== 7598+0+0 (unknown 1739579013 0 0) 0x7f09f0078ae0 con 0x7f09f41042a0 2024-02-18T01:36:57.944 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.940+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=726}) v3 -- 0x7f09f411a5d0 con 0x7f09f41042a0 2024-02-18T01:36:57.950 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:57.948+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f09f004c100 con 0x7f09f41042a0 2024-02-18T01:36:58.122 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:58.120+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f09d8005190 con 0x7f09f41042a0 2024-02-18T01:36:58.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:36:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:36:59.126 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.124+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 10 ==== osd_map(726..726 src has 1..726) v4 ==== 620+0+0 (unknown 3534465775 0 0) 0x7f09f004c330 con 0x7f09f41042a0 2024-02-18T01:36:59.126 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.124+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=727}) v3 -- 0x7f09f411b770 con 0x7f09f41042a0 2024-02-18T01:36:59.131 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.128+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v726) v1 ==== 225+0+0 (unknown 3530292927 0 0) 0x7f09f004a510 con 0x7f09f41042a0 2024-02-18T01:36:59.251 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.248+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f09d8002220 con 0x7f09f41042a0 2024-02-18T01:36:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:59 smithi105 bash[19734]: cluster 2024-02-18T01:36:58.053775+0000 mgr.y (mgr.24475) 764 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:36:59 smithi105 bash[19734]: audit 2024-02-18T01:36:58.122727+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:36:59.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:59 smithi105 bash[27669]: cluster 2024-02-18T01:36:58.053775+0000 mgr.y (mgr.24475) 764 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:59.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:36:59 smithi105 bash[27669]: audit 2024-02-18T01:36:58.122727+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:36:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:59 smithi155 bash[20805]: cluster 2024-02-18T01:36:58.053775+0000 mgr.y (mgr.24475) 764 : cluster [DBG] pgmap v1248: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:36:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:36:59 smithi155 bash[20805]: audit 2024-02-18T01:36:58.122727+0000 mon.a (mon.0) 3234 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:36:59.662 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.660+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 12 ==== osd_map(727..727 src has 1..727) v4 ==== 620+0+0 (unknown 3207525376 0 0) 0x7f09f0051c80 con 0x7f09f41042a0 2024-02-18T01:36:59.662 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.660+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=728}) v3 -- 0x7f09e006ede0 con 0x7f09f41042a0 2024-02-18T01:36:59.669 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.668+0000 7f09eaffd700 1 -- 172.21.15.105:0/819780126 <== mon.0 v1:172.21.15.105:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v727) v1 ==== 225+0+0 (unknown 4203266759 0 0) 0x7f09f004bc90 con 0x7f09f41042a0 2024-02-18T01:36:59.673 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.668+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 >> v1:172.21.15.105:6800/2148176012 conn(0x7f09e0062340 legacy=0x7f09e0064800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:59.673 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.668+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 >> v1:172.21.15.105:6789/0 conn(0x7f09f41042a0 legacy=0x7f09f4117d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:59.674 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.672+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 shutdown_connections 2024-02-18T01:36:59.675 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.672+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 >> 172.21.15.105:0/819780126 conn(0x7f09f40f24d0 msgr2=0x7f09f4107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:59.675 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.672+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 shutdown_connections 2024-02-18T01:36:59.675 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.672+0000 7f09fbf89700 1 -- 172.21.15.105:0/819780126 wait complete. 2024-02-18T01:36:59.675 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_objects = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:36:59.687 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 128407 2024-02-18T01:36:59.687 INFO:tasks.workunit.client.0.smithi105.stderr:+ [ 0 -ne 0 ] 2024-02-18T01:36:59.687 INFO:tasks.workunit.client.0.smithi105.stderr:+ true 2024-02-18T01:36:59.687 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put twomore /etc/passwd 2024-02-18T01:36:59.725 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_bytes 100 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.844+0000 7f7bc1264700 1 -- 172.21.15.105:0/3714801217 >> v1:172.21.15.105:6790/0 conn(0x7f7bbc1007f0 legacy=0x7f7bbc102be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/3714801217 shutdown_connections 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/3714801217 >> 172.21.15.105:0/3714801217 conn(0x7f7bbc06c100 msgr2=0x7f7bbc06c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/3714801217 shutdown_connections 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/3714801217 wait complete. 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 Processor -- start 2024-02-18T01:36:59.850 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- start start 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7bbc117340 con 0x7f7bbc104300 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7bbc118520 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f7bbc119720 con 0x7f7bbc1007f0 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bba7fc700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f7bbc1007f0 0x7f7bbc0fa490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:53436/0 (socket says 172.21.15.105:53436) 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bba7fc700 1 -- 172.21.15.105:0/2469881073 learned_addr learned my addr 172.21.15.105:0/2469881073 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889055971 0 0) 0x7f7bbc119720 con 0x7f7bbc1007f0 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ba40035f0 con 0x7f7bbc1007f0 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346482828 0 0) 0x7f7bbc117340 con 0x7f7bbc104300 2024-02-18T01:36:59.853 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc119720 con 0x7f7bbc104300 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 272971219 0 0) 0x7f7bbc118520 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bbc117340 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 644581019 0 0) 0x7f7ba40035f0 con 0x7f7bbc1007f0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bbc118520 con 0x7f7bbc1007f0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7bac003200 con 0x7f7bbc1007f0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4274142698 0 0) 0x7f7bbc119720 con 0x7f7bbc104300 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7ba40035f0 con 0x7f7bbc104300 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7bb4001670 con 0x7f7bbc104300 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2242750263 0 0) 0x7f7bbc117340 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f7bbc119720 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7bb00019d0 con 0x7f7bbc0f62e0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2772441933 0 0) 0x7f7bbc118520 con 0x7f7bbc1007f0 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 >> v1:172.21.15.155:6789/0 conn(0x7f7bbc0f62e0 legacy=0x7f7bbc0f9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:59.854 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 >> v1:172.21.15.105:6789/0 conn(0x7f7bbc104300 legacy=0x7f7bbc115a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:36:59.855 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc11a920 con 0x7f7bbc1007f0 2024-02-18T01:36:59.855 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc117530 con 0x7f7bbc1007f0 2024-02-18T01:36:59.855 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc117980 con 0x7f7bbc1007f0 2024-02-18T01:36:59.855 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f7bac001e70 con 0x7f7bbc1007f0 2024-02-18T01:36:59.855 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.848+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f7bac003c80 con 0x7f7bbc1007f0 2024-02-18T01:36:59.856 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.852+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f7bac003de0 con 0x7f7bbc1007f0 2024-02-18T01:36:59.856 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.852+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ba8004ee0 con 0x7f7bbc1007f0 2024-02-18T01:36:59.856 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.852+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(727..727 src has 1..727) v4 ==== 7598+0+0 (unknown 3576246207 0 0) 0x7f7bac079200 con 0x7f7bbc1007f0 2024-02-18T01:36:59.861 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:36:59.860+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f7bac04c820 con 0x7f7bbc1007f0 2024-02-18T01:37:00.041 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:00.040+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f7ba8005190 con 0x7f7bbc1007f0 2024-02-18T01:37:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:36:58.303966+0000 mgr.y (mgr.24475) 765 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:36:59.131237+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: cluster 2024-02-18T01:36:59.131304+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T01:37:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:36:59.252166+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: cluster 2024-02-18T01:36:59.658524+0000 mon.a (mon.0) 3238 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: cluster 2024-02-18T01:36:59.659014+0000 mon.a (mon.0) 3239 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:36:59.668819+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: cluster 2024-02-18T01:36:59.669027+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:37:00.041722+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[19734]: audit 2024-02-18T01:37:00.042223+0000 mon.a (mon.0) 3242 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:36:58.303966+0000 mgr.y (mgr.24475) 765 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:36:59.131237+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: cluster 2024-02-18T01:36:59.131304+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:36:59.252166+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:37:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: cluster 2024-02-18T01:36:59.658524+0000 mon.a (mon.0) 3238 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: cluster 2024-02-18T01:36:59.659014+0000 mon.a (mon.0) 3239 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:36:59.668819+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: cluster 2024-02-18T01:36:59.669027+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T01:37:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:37:00.041722+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.491 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:00 smithi105 bash[27669]: audit 2024-02-18T01:37:00.042223+0000 mon.a (mon.0) 3242 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:36:58.303966+0000 mgr.y (mgr.24475) 765 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:36:59.131237+0000 mon.a (mon.0) 3235 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: cluster 2024-02-18T01:36:59.131304+0000 mon.a (mon.0) 3236 : cluster [DBG] osdmap e726: 8 total, 8 up, 8 in 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:36:59.252166+0000 mon.a (mon.0) 3237 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]: dispatch 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: cluster 2024-02-18T01:36:59.658524+0000 mon.a (mon.0) 3238 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: cluster 2024-02-18T01:36:59.659014+0000 mon.a (mon.0) 3239 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:36:59.668819+0000 mon.a (mon.0) 3240 : audit [INF] from='client.? v1:172.21.15.105:0/819780126' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "100"}]': finished 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: cluster 2024-02-18T01:36:59.669027+0000 mon.a (mon.0) 3241 : cluster [DBG] osdmap e727: 8 total, 8 up, 8 in 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:37:00.041722+0000 mon.c (mon.2) 419 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:00 smithi155 bash[20805]: audit 2024-02-18T01:37:00.042223+0000 mon.a (mon.0) 3242 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:00.681 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:00.680+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v728) v1 ==== 221+0+0 (unknown 2648869322 0 0) 0x7f7bac04ac90 con 0x7f7bbc1007f0 2024-02-18T01:37:00.802 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:00.800+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f7ba80058d0 con 0x7f7bbc1007f0 2024-02-18T01:37:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[19734]: cluster 2024-02-18T01:37:00.054479+0000 mgr.y (mgr.24475) 766 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[19734]: audit 2024-02-18T01:37:00.680815+0000 mon.a (mon.0) 3243 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[19734]: cluster 2024-02-18T01:37:00.681052+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T01:37:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[19734]: audit 2024-02-18T01:37:00.802365+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[19734]: audit 2024-02-18T01:37:00.803101+0000 mon.a (mon.0) 3245 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[27669]: cluster 2024-02-18T01:37:00.054479+0000 mgr.y (mgr.24475) 766 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[27669]: audit 2024-02-18T01:37:00.680815+0000 mon.a (mon.0) 3243 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[27669]: cluster 2024-02-18T01:37:00.681052+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[27669]: audit 2024-02-18T01:37:00.802365+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:01 smithi105 bash[27669]: audit 2024-02-18T01:37:00.803101+0000 mon.a (mon.0) 3245 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[20805]: cluster 2024-02-18T01:37:00.054479+0000 mgr.y (mgr.24475) 766 : cluster [DBG] pgmap v1251: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[20805]: audit 2024-02-18T01:37:00.680815+0000 mon.a (mon.0) 3243 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[20805]: cluster 2024-02-18T01:37:00.681052+0000 mon.a (mon.0) 3244 : cluster [DBG] osdmap e728: 8 total, 8 up, 8 in 2024-02-18T01:37:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[20805]: audit 2024-02-18T01:37:00.802365+0000 mon.c (mon.2) 420 : audit [INF] from='client.? v1:172.21.15.105:0/2469881073' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[20805]: audit 2024-02-18T01:37:00.803101+0000 mon.a (mon.0) 3245 : audit [INF] from='client.48401 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]: dispatch 2024-02-18T01:37:01.682 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7ba3fff700 1 -- 172.21.15.105:0/2469881073 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v729) v1 ==== 221+0+0 (unknown 498227243 0 0) 0x7f7bac04c410 con 0x7f7bbc1007f0 2024-02-18T01:37:01.684 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 >> v1:172.21.15.105:6800/2148176012 conn(0x7f7ba40622d0 legacy=0x7f7ba4064790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:01.684 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 >> v1:172.21.15.105:6790/0 conn(0x7f7bbc1007f0 legacy=0x7f7bbc0fa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:01.684 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 shutdown_connections 2024-02-18T01:37:01.684 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 >> 172.21.15.105:0/2469881073 conn(0x7f7bbc06c100 msgr2=0x7f7bbc06cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:01.685 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 shutdown_connections 2024-02-18T01:37:01.685 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:01.680+0000 7f7bc1264700 1 -- 172.21.15.105:0/2469881073 wait complete. 2024-02-18T01:37:01.685 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_bytes = 100 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:37:01.698 INFO:tasks.workunit.client.0.smithi105.stderr:+ sleep 30 2024-02-18T01:37:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:02.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[19734]: audit 2024-02-18T01:37:01.679224+0000 mon.a (mon.0) 3246 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:02.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[19734]: cluster 2024-02-18T01:37:01.679379+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T01:37:02.992 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[19734]: cluster 2024-02-18T01:37:02.055293+0000 mgr.y (mgr.24475) 767 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:37:02.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[27669]: audit 2024-02-18T01:37:01.679224+0000 mon.a (mon.0) 3246 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:02.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[27669]: cluster 2024-02-18T01:37:01.679379+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T01:37:02.992 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:02 smithi105 bash[27669]: cluster 2024-02-18T01:37:02.055293+0000 mgr.y (mgr.24475) 767 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:37:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:02 smithi155 bash[20805]: audit 2024-02-18T01:37:01.679224+0000 mon.a (mon.0) 3246 : audit [INF] from='client.48401 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "100"}]': finished 2024-02-18T01:37:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:02 smithi155 bash[20805]: cluster 2024-02-18T01:37:01.679379+0000 mon.a (mon.0) 3247 : cluster [DBG] osdmap e729: 8 total, 8 up, 8 in 2024-02-18T01:37:02.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:02 smithi155 bash[20805]: cluster 2024-02-18T01:37:02.055293+0000 mgr.y (mgr.24475) 767 : cluster [DBG] pgmap v1254: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-02-18T01:37:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[19734]: cluster 2024-02-18T01:37:04.057282+0000 mgr.y (mgr.24475) 768 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 830 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:37:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[19734]: cluster 2024-02-18T01:37:04.660743+0000 mon.a (mon.0) 3248 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_bytes: 100 B) 2024-02-18T01:37:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[19734]: cluster 2024-02-18T01:37:04.661257+0000 mon.a (mon.0) 3249 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:37:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[19734]: cluster 2024-02-18T01:37:04.670158+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T01:37:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[27669]: cluster 2024-02-18T01:37:04.057282+0000 mgr.y (mgr.24475) 768 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 830 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:37:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[27669]: cluster 2024-02-18T01:37:04.660743+0000 mon.a (mon.0) 3248 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_bytes: 100 B) 2024-02-18T01:37:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[27669]: cluster 2024-02-18T01:37:04.661257+0000 mon.a (mon.0) 3249 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:37:05.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:05 smithi105 bash[27669]: cluster 2024-02-18T01:37:04.670158+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T01:37:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:05 smithi155 bash[20805]: cluster 2024-02-18T01:37:04.057282+0000 mgr.y (mgr.24475) 768 : cluster [DBG] pgmap v1255: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 830 B/s rd, 4.9 KiB/s wr, 1 op/s 2024-02-18T01:37:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:05 smithi155 bash[20805]: cluster 2024-02-18T01:37:04.660743+0000 mon.a (mon.0) 3248 : cluster [WRN] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' is full (reached quota's max_bytes: 100 B) 2024-02-18T01:37:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:05 smithi155 bash[20805]: cluster 2024-02-18T01:37:04.661257+0000 mon.a (mon.0) 3249 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:37:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:05 smithi155 bash[20805]: cluster 2024-02-18T01:37:04.670158+0000 mon.a (mon.0) 3250 : cluster [DBG] osdmap e730: 8 total, 8 up, 8 in 2024-02-18T01:37:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:06] "GET /metrics HTTP/1.1" 200 189297 "" "Prometheus/2.33.4" 2024-02-18T01:37:07.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:07 smithi105 bash[19734]: cluster 2024-02-18T01:37:06.058160+0000 mgr.y (mgr.24475) 769 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 4.0 KiB/s wr, 0 op/s 2024-02-18T01:37:07.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:07 smithi105 bash[27669]: cluster 2024-02-18T01:37:06.058160+0000 mgr.y (mgr.24475) 769 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 4.0 KiB/s wr, 0 op/s 2024-02-18T01:37:07.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:07 smithi155 bash[20805]: cluster 2024-02-18T01:37:06.058160+0000 mgr.y (mgr.24475) 769 : cluster [DBG] pgmap v1257: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 4.0 KiB/s wr, 0 op/s 2024-02-18T01:37:08.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:09 smithi105 bash[19734]: cluster 2024-02-18T01:37:08.060033+0000 mgr.y (mgr.24475) 770 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-18T01:37:09.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:09 smithi105 bash[27669]: cluster 2024-02-18T01:37:08.060033+0000 mgr.y (mgr.24475) 770 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-18T01:37:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:09 smithi155 bash[20805]: cluster 2024-02-18T01:37:08.060033+0000 mgr.y (mgr.24475) 770 : cluster [DBG] pgmap v1258: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.2 KiB/s wr, 1 op/s 2024-02-18T01:37:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:10 smithi105 bash[19734]: audit 2024-02-18T01:37:08.315395+0000 mgr.y (mgr.24475) 771 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:10 smithi105 bash[27669]: audit 2024-02-18T01:37:08.315395+0000 mgr.y (mgr.24475) 771 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:10 smithi155 bash[20805]: audit 2024-02-18T01:37:08.315395+0000 mgr.y (mgr.24475) 771 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:11 smithi105 bash[19734]: cluster 2024-02-18T01:37:10.060938+0000 mgr.y (mgr.24475) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-18T01:37:11.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:11 smithi105 bash[27669]: cluster 2024-02-18T01:37:10.060938+0000 mgr.y (mgr.24475) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-18T01:37:11.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:11 smithi155 bash[20805]: cluster 2024-02-18T01:37:10.060938+0000 mgr.y (mgr.24475) 772 : cluster [DBG] pgmap v1259: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-02-18T01:37:12.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:13.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:13 smithi105 bash[27669]: cluster 2024-02-18T01:37:12.061883+0000 mgr.y (mgr.24475) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-18T01:37:13.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:13 smithi105 bash[19734]: cluster 2024-02-18T01:37:12.061883+0000 mgr.y (mgr.24475) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-18T01:37:13.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:13 smithi155 bash[20805]: cluster 2024-02-18T01:37:12.061883+0000 mgr.y (mgr.24475) 773 : cluster [DBG] pgmap v1260: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 2.4 KiB/s wr, 1 op/s 2024-02-18T01:37:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:15 smithi105 bash[19734]: cluster 2024-02-18T01:37:14.063902+0000 mgr.y (mgr.24475) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:37:15.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:15 smithi105 bash[27669]: cluster 2024-02-18T01:37:14.063902+0000 mgr.y (mgr.24475) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:37:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:15 smithi155 bash[20805]: cluster 2024-02-18T01:37:14.063902+0000 mgr.y (mgr.24475) 774 : cluster [DBG] pgmap v1261: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:37:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:16 smithi105 bash[27669]: cluster 2024-02-18T01:37:16.064707+0000 mgr.y (mgr.24475) 775 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-02-18T01:37:16.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:16 smithi105 bash[19734]: cluster 2024-02-18T01:37:16.064707+0000 mgr.y (mgr.24475) 775 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-02-18T01:37:16.496 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:16 smithi155 bash[20805]: cluster 2024-02-18T01:37:16.064707+0000 mgr.y (mgr.24475) 775 : cluster [DBG] pgmap v1262: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 898 B/s rd, 0 op/s 2024-02-18T01:37:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:16] "GET /metrics HTTP/1.1" 200 189297 "" "Prometheus/2.33.4" 2024-02-18T01:37:18.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:19 smithi105 bash[19734]: cluster 2024-02-18T01:37:18.065788+0000 mgr.y (mgr.24475) 776 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:19 smithi105 bash[27669]: cluster 2024-02-18T01:37:18.065788+0000 mgr.y (mgr.24475) 776 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:19.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:19 smithi155 bash[20805]: cluster 2024-02-18T01:37:18.065788+0000 mgr.y (mgr.24475) 776 : cluster [DBG] pgmap v1263: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:20.492 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:20 smithi105 bash[19734]: audit 2024-02-18T01:37:18.317363+0000 mgr.y (mgr.24475) 777 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:20.492 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:20 smithi105 bash[27669]: audit 2024-02-18T01:37:18.317363+0000 mgr.y (mgr.24475) 777 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:20 smithi155 bash[20805]: audit 2024-02-18T01:37:18.317363+0000 mgr.y (mgr.24475) 777 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:21.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:21 smithi105 bash[27669]: cluster 2024-02-18T01:37:20.066542+0000 mgr.y (mgr.24475) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:21.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:21 smithi105 bash[19734]: cluster 2024-02-18T01:37:20.066542+0000 mgr.y (mgr.24475) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:21.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:21 smithi155 bash[20805]: cluster 2024-02-18T01:37:20.066542+0000 mgr.y (mgr.24475) 778 : cluster [DBG] pgmap v1264: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:23.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:23 smithi105 bash[19734]: cluster 2024-02-18T01:37:22.067318+0000 mgr.y (mgr.24475) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:23.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:23 smithi105 bash[27669]: cluster 2024-02-18T01:37:22.067318+0000 mgr.y (mgr.24475) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:23.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:23 smithi155 bash[20805]: cluster 2024-02-18T01:37:22.067318+0000 mgr.y (mgr.24475) 779 : cluster [DBG] pgmap v1265: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:25.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:25 smithi105 bash[19734]: cluster 2024-02-18T01:37:24.069221+0000 mgr.y (mgr.24475) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:25.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:25 smithi105 bash[27669]: cluster 2024-02-18T01:37:24.069221+0000 mgr.y (mgr.24475) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:25.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:25 smithi155 bash[20805]: cluster 2024-02-18T01:37:24.069221+0000 mgr.y (mgr.24475) 780 : cluster [DBG] pgmap v1266: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:26.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:26 smithi105 bash[19734]: cluster 2024-02-18T01:37:26.070000+0000 mgr.y (mgr.24475) 781 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:26.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:26 smithi105 bash[27669]: cluster 2024-02-18T01:37:26.070000+0000 mgr.y (mgr.24475) 781 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:26.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:26 smithi155 bash[20805]: cluster 2024-02-18T01:37:26.070000+0000 mgr.y (mgr.24475) 781 : cluster [DBG] pgmap v1267: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:26] "GET /metrics HTTP/1.1" 200 189306 "" "Prometheus/2.33.4" 2024-02-18T01:37:28.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:28 smithi105 bash[19734]: audit 2024-02-18T01:37:28.082396+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:37:28.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:28 smithi105 bash[27669]: audit 2024-02-18T01:37:28.082396+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:37:28.492 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:28 smithi155 bash[20805]: audit 2024-02-18T01:37:28.082396+0000 mon.a (mon.0) 3251 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:37:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[19734]: cluster 2024-02-18T01:37:28.071844+0000 mgr.y (mgr.24475) 782 : cluster [DBG] pgmap v1268: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[19734]: audit 2024-02-18T01:37:28.400963+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:37:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[19734]: audit 2024-02-18T01:37:28.402669+0000 mon.a (mon.0) 3253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:37:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[19734]: audit 2024-02-18T01:37:28.414388+0000 mon.a (mon.0) 3254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[27669]: cluster 2024-02-18T01:37:28.071844+0000 mgr.y (mgr.24475) 782 : cluster [DBG] pgmap v1268: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[27669]: audit 2024-02-18T01:37:28.400963+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:37:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[27669]: audit 2024-02-18T01:37:28.402669+0000 mon.a (mon.0) 3253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:37:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:29 smithi105 bash[27669]: audit 2024-02-18T01:37:28.414388+0000 mon.a (mon.0) 3254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:29 smithi155 bash[20805]: cluster 2024-02-18T01:37:28.071844+0000 mgr.y (mgr.24475) 782 : cluster [DBG] pgmap v1268: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:29 smithi155 bash[20805]: audit 2024-02-18T01:37:28.400963+0000 mon.a (mon.0) 3252 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:37:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:29 smithi155 bash[20805]: audit 2024-02-18T01:37:28.402669+0000 mon.a (mon.0) 3253 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:37:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:29 smithi155 bash[20805]: audit 2024-02-18T01:37:28.414388+0000 mon.a (mon.0) 3254 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:30.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:30 smithi105 bash[19734]: audit 2024-02-18T01:37:28.325636+0000 mgr.y (mgr.24475) 783 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:30.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:30 smithi105 bash[27669]: audit 2024-02-18T01:37:28.325636+0000 mgr.y (mgr.24475) 783 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:30.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:30 smithi155 bash[20805]: audit 2024-02-18T01:37:28.325636+0000 mgr.y (mgr.24475) 783 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:31.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:31 smithi105 bash[19734]: cluster 2024-02-18T01:37:30.072618+0000 mgr.y (mgr.24475) 784 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:31.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:31 smithi105 bash[27669]: cluster 2024-02-18T01:37:30.072618+0000 mgr.y (mgr.24475) 784 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:31.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:31 smithi155 bash[20805]: cluster 2024-02-18T01:37:30.072618+0000 mgr.y (mgr.24475) 784 : cluster [DBG] pgmap v1269: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:31.700 INFO:tasks.workunit.client.0.smithi105.stderr:+ pid=128508 2024-02-18T01:37:31.700 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_bytes 0 2024-02-18T01:37:31.701 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put two /etc/passwd 2024-02-18T01:37:31.835 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- 172.21.15.105:0/2536417233 >> v1:172.21.15.105:6789/0 conn(0x7f47100f8940 legacy=0x7f47100f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:31.835 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- 172.21.15.105:0/2536417233 shutdown_connections 2024-02-18T01:37:31.836 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- 172.21.15.105:0/2536417233 >> 172.21.15.105:0/2536417233 conn(0x7f47100f24d0 msgr2=0x7f47100f48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:31.836 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- 172.21.15.105:0/2536417233 shutdown_connections 2024-02-18T01:37:31.836 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- 172.21.15.105:0/2536417233 wait complete. 2024-02-18T01:37:31.836 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 Processor -- start 2024-02-18T01:37:31.838 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- start start 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47101173d0 con 0x7f47101042a0 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47101185b0 con 0x7f47100f8940 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4717538700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f47101197b0 con 0x7f47100f9430 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4714ad3700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f47100f9430 0x7f47100ffe00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:35180/0 (socket says 172.21.15.105:35180) 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f4714ad3700 1 -- 172.21.15.105:0/2685087116 learned_addr learned my addr 172.21.15.105:0/2685087116 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918885998 0 0) 0x7f47101173d0 con 0x7f47101042a0 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46f80033b0 con 0x7f47101042a0 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644775470 0 0) 0x7f47101197b0 con 0x7f47100f9430 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47101173d0 con 0x7f47100f9430 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2092420136 0 0) 0x7f47101185b0 con 0x7f47100f8940 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.832+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47101197b0 con 0x7f47100f8940 2024-02-18T01:37:31.839 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2264539194 0 0) 0x7f46f80033b0 con 0x7f47101042a0 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47101185b0 con 0x7f47101042a0 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f47000017b0 con 0x7f47101042a0 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2386148112 0 0) 0x7f47101173d0 con 0x7f47100f9430 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f46f80033b0 con 0x7f47100f9430 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 963236903 0 0) 0x7f47101197b0 con 0x7f47100f8940 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f47101173d0 con 0x7f47100f8940 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4704001900 con 0x7f47100f9430 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4708002fb0 con 0x7f47100f8940 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 109167135 0 0) 0x7f47101185b0 con 0x7f47101042a0 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 >> v1:172.21.15.105:6790/0 conn(0x7f47100f9430 legacy=0x7f47100ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 >> v1:172.21.15.155:6789/0 conn(0x7f47100f8940 legacy=0x7f47100ff6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471011a9b0 con 0x7f47101042a0 2024-02-18T01:37:31.840 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47101175c0 con 0x7f47101042a0 2024-02-18T01:37:31.841 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4710117a10 con 0x7f47101042a0 2024-02-18T01:37:31.841 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4700001ec0 con 0x7f47101042a0 2024-02-18T01:37:31.841 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4700003210 con 0x7f47101042a0 2024-02-18T01:37:31.841 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46fc004ee0 con 0x7f47101042a0 2024-02-18T01:37:31.841 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f4700016180 con 0x7f47101042a0 2024-02-18T01:37:31.842 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(730..730 src has 1..730) v4 ==== 7598+0+0 (unknown 759753389 0 0) 0x7f4700076320 con 0x7f47101042a0 2024-02-18T01:37:31.842 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.836+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=731}) v3 -- 0x7f47101185b0 con 0x7f47101042a0 2024-02-18T01:37:31.848 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:31.844+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f47000498e0 con 0x7f47101042a0 2024-02-18T01:37:32.030 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:32.028+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f46fc005190 con 0x7f47101042a0 2024-02-18T01:37:32.166 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:32.164+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 10 ==== osd_map(731..731 src has 1..731) v4 ==== 620+0+0 (unknown 774804955 0 0) 0x7f4700049b10 con 0x7f47101042a0 2024-02-18T01:37:32.166 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:32.164+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=732}) v3 -- 0x7f47101197b0 con 0x7f47101042a0 2024-02-18T01:37:32.168 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:32.172 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:32.168+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v731) v1 ==== 217+0+0 (unknown 2022378503 0 0) 0x7f4700047cf0 con 0x7f47101042a0 2024-02-18T01:37:32.288 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:32.284+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f46fc005800 con 0x7f47101042a0 2024-02-18T01:37:32.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:32 smithi105 bash[19734]: audit 2024-02-18T01:37:32.030243+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:32.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:32 smithi105 bash[19734]: cluster 2024-02-18T01:37:32.073366+0000 mgr.y (mgr.24475) 785 : cluster [DBG] pgmap v1270: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:32.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:32 smithi105 bash[27669]: audit 2024-02-18T01:37:32.030243+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:32.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:32 smithi105 bash[27669]: cluster 2024-02-18T01:37:32.073366+0000 mgr.y (mgr.24475) 785 : cluster [DBG] pgmap v1270: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:32.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:32 smithi155 bash[20805]: audit 2024-02-18T01:37:32.030243+0000 mon.a (mon.0) 3255 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:32.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:32 smithi155 bash[20805]: cluster 2024-02-18T01:37:32.073366+0000 mgr.y (mgr.24475) 785 : cluster [DBG] pgmap v1270: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-02-18T01:37:33.168 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.164+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 12 ==== osd_map(732..732 src has 1..732) v4 ==== 620+0+0 (unknown 1781515125 0 0) 0x7f4700047eb0 con 0x7f47101042a0 2024-02-18T01:37:33.169 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.164+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=733}) v3 -- 0x7f46f806ee10 con 0x7f47101042a0 2024-02-18T01:37:33.175 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.172+0000 7f470e7fc700 1 -- 172.21.15.105:0/2685087116 <== mon.0 v1:172.21.15.105:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v732) v1 ==== 217+0+0 (unknown 1171439759 0 0) 0x7f4700049470 con 0x7f47101042a0 2024-02-18T01:37:33.180 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.176+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 >> v1:172.21.15.105:6800/2148176012 conn(0x7f46f8062380 legacy=0x7f46f8064840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:33.180 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.176+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 >> v1:172.21.15.105:6789/0 conn(0x7f47101042a0 legacy=0x7f4710115a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:33.180 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.176+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 shutdown_connections 2024-02-18T01:37:33.180 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.176+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 >> 172.21.15.105:0/2685087116 conn(0x7f47100f24d0 msgr2=0x7f4710107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:33.180 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.176+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 shutdown_connections 2024-02-18T01:37:33.181 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.180+0000 7f4717538700 1 -- 172.21.15.105:0/2685087116 wait complete. 2024-02-18T01:37:33.181 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:37:33.195 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_objects 0 2024-02-18T01:37:33.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/2182808494 >> v1:172.21.15.105:6790/0 conn(0x7fad5c102760 legacy=0x7fad5c104bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:33.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/2182808494 shutdown_connections 2024-02-18T01:37:33.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/2182808494 >> 172.21.15.105:0/2182808494 conn(0x7fad5c06cb00 msgr2=0x7fad5c0f3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:33.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/2182808494 shutdown_connections 2024-02-18T01:37:33.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/2182808494 wait complete. 2024-02-18T01:37:33.319 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 Processor -- start 2024-02-18T01:37:33.319 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- start start 2024-02-18T01:37:33.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad5c0fb770 con 0x7fad5c0fec50 2024-02-18T01:37:33.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad61b04700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fad5c0fec50 0x7fad5c0fa360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41408/0 (socket says 172.21.15.105:41408) 2024-02-18T01:37:33.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad5c118500 con 0x7fad5c102760 2024-02-18T01:37:33.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad61b04700 1 -- 172.21.15.105:0/1906352619 learned_addr learned my addr 172.21.15.105:0/1906352619 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:37:33.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fad5c1196a0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1874595893 0 0) 0x7fad5c1196a0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad44004700 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 13405657 0 0) 0x7fad5c0fb770 con 0x7fad5c0fec50 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad5c1196a0 con 0x7fad5c0fec50 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3951081950 0 0) 0x7fad5c118500 con 0x7fad5c102760 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.316+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad5c0fb770 con 0x7fad5c102760 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3950024495 0 0) 0x7fad44004700 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad5c118500 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fad4c001c00 con 0x7fad5c0fbde0 2024-02-18T01:37:33.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 215943623 0 0) 0x7fad5c1196a0 con 0x7fad5c0fec50 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fad44004700 con 0x7fad5c0fec50 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fad500018c0 con 0x7fad5c0fec50 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1003993440 0 0) 0x7fad5c118500 con 0x7fad5c0fbde0 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 >> v1:172.21.15.155:6789/0 conn(0x7fad5c102760 legacy=0x7fad5c115dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 >> v1:172.21.15.105:6789/0 conn(0x7fad5c0fec50 legacy=0x7fad5c0fa360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c11a840 con 0x7fad5c0fbde0 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c119890 con 0x7fad5c0fbde0 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c119ce0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fad4c002fc0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fad4c003650 con 0x7fad5c0fbde0 2024-02-18T01:37:33.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad48004ee0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fad4c002120 con 0x7fad5c0fbde0 2024-02-18T01:37:33.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(732..732 src has 1..732) v4 ==== 7598+0+0 (unknown 4281306898 0 0) 0x7fad4c076810 con 0x7fad5c0fbde0 2024-02-18T01:37:33.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.320+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=733}) v3 -- 0x7fad5c118500 con 0x7fad5c0fbde0 2024-02-18T01:37:33.329 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.328+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fad4c049dd0 con 0x7fad5c0fbde0 2024-02-18T01:37:33.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[19734]: audit 2024-02-18T01:37:32.171618+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:33.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[19734]: cluster 2024-02-18T01:37:32.171685+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T01:37:33.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[19734]: audit 2024-02-18T01:37:32.288329+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:33.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[27669]: audit 2024-02-18T01:37:32.171618+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[27669]: cluster 2024-02-18T01:37:32.171685+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T01:37:33.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:33 smithi105 bash[27669]: audit 2024-02-18T01:37:32.288329+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:33.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:33 smithi155 bash[20805]: audit 2024-02-18T01:37:32.171618+0000 mon.a (mon.0) 3256 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:33.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:33 smithi155 bash[20805]: cluster 2024-02-18T01:37:32.171685+0000 mon.a (mon.0) 3257 : cluster [DBG] osdmap e731: 8 total, 8 up, 8 in 2024-02-18T01:37:33.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:33 smithi155 bash[20805]: audit 2024-02-18T01:37:32.288329+0000 mon.a (mon.0) 3258 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:37:33.511 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:33.508+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fad48005190 con 0x7fad5c0fbde0 2024-02-18T01:37:34.198 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.196+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 10 ==== osd_map(733..733 src has 1..733) v4 ==== 620+0+0 (unknown 3476400393 0 0) 0x7fad4c04a000 con 0x7fad5c0fbde0 2024-02-18T01:37:34.198 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.196+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=734}) v3 -- 0x7fad5c1196a0 con 0x7fad5c0fbde0 2024-02-18T01:37:34.205 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.204+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v733) v1 ==== 221+0+0 (unknown 1880322723 0 0) 0x7fad4c0481e0 con 0x7fad5c0fbde0 2024-02-18T01:37:34.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.320+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fad48005800 con 0x7fad5c0fbde0 2024-02-18T01:37:34.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[27669]: audit 2024-02-18T01:37:33.175324+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:34.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[27669]: cluster 2024-02-18T01:37:33.175697+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T01:37:34.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[27669]: audit 2024-02-18T01:37:33.511913+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[27669]: audit 2024-02-18T01:37:33.512624+0000 mon.a (mon.0) 3261 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[27669]: cluster 2024-02-18T01:37:34.074168+0000 mgr.y (mgr.24475) 786 : cluster [DBG] pgmap v1273: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[19734]: audit 2024-02-18T01:37:33.175324+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[19734]: cluster 2024-02-18T01:37:33.175697+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[19734]: audit 2024-02-18T01:37:33.511913+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[19734]: audit 2024-02-18T01:37:33.512624+0000 mon.a (mon.0) 3261 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:34 smithi105 bash[19734]: cluster 2024-02-18T01:37:34.074168+0000 mgr.y (mgr.24475) 786 : cluster [DBG] pgmap v1273: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:34 smithi155 bash[20805]: audit 2024-02-18T01:37:33.175324+0000 mon.a (mon.0) 3259 : audit [INF] from='client.? v1:172.21.15.105:0/2685087116' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:37:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:34 smithi155 bash[20805]: cluster 2024-02-18T01:37:33.175697+0000 mon.a (mon.0) 3260 : cluster [DBG] osdmap e732: 8 total, 8 up, 8 in 2024-02-18T01:37:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:34 smithi155 bash[20805]: audit 2024-02-18T01:37:33.511913+0000 mon.c (mon.2) 421 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:34 smithi155 bash[20805]: audit 2024-02-18T01:37:33.512624+0000 mon.a (mon.0) 3261 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:34 smithi155 bash[20805]: cluster 2024-02-18T01:37:34.074168+0000 mgr.y (mgr.24475) 786 : cluster [DBG] pgmap v1273: 176 pgs: 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:34.676 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.672+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 12 ==== osd_map(734..734 src has 1..734) v4 ==== 620+0+0 (unknown 1395931839 0 0) 0x7fad4c04f950 con 0x7fad5c0fbde0 2024-02-18T01:37:34.676 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.672+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=735}) v3 -- 0x7fad4406ec50 con 0x7fad5c0fbde0 2024-02-18T01:37:34.682 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.680+0000 7fad5b7fe700 1 -- 172.21.15.105:0/1906352619 <== mon.2 v1:172.21.15.105:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v734) v1 ==== 221+0+0 (unknown 4111365745 0 0) 0x7fad4c049960 con 0x7fad5c0fbde0 2024-02-18T01:37:34.688 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 >> v1:172.21.15.105:6800/2148176012 conn(0x7fad44062120 legacy=0x7fad440645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:34.688 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 >> v1:172.21.15.105:6790/0 conn(0x7fad5c0fbde0 legacy=0x7fad5c0f9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:34.688 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 shutdown_connections 2024-02-18T01:37:34.688 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 >> 172.21.15.105:0/1906352619 conn(0x7fad5c06cb00 msgr2=0x7fad5c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:34.689 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 shutdown_connections 2024-02-18T01:37:34.689 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.684+0000 7fad63307700 1 -- 172.21.15.105:0/1906352619 wait complete. 2024-02-18T01:37:34.689 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:37:34.697 INFO:tasks.workunit.client.0.smithi105.stderr:+ wait 128508 2024-02-18T01:37:34.697 INFO:tasks.workunit.client.0.smithi105.stderr:+ [ 0 -ne 0 ] 2024-02-18T01:37:34.698 INFO:tasks.workunit.client.0.smithi105.stderr:+ true 2024-02-18T01:37:34.698 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put three /etc/passwd 2024-02-18T01:37:34.737 INFO:tasks.workunit.client.0.smithi105.stderr:+ uuidgen 2024-02-18T01:37:34.738 INFO:tasks.workunit.client.0.smithi105.stderr:+ pp=7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 2024-02-18T01:37:34.738 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool create 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 12 2024-02-18T01:37:34.858 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- 172.21.15.105:0/3687633712 >> v1:172.21.15.105:6790/0 conn(0x7efd981042a0 legacy=0x7efd98106740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:34.859 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- 172.21.15.105:0/3687633712 shutdown_connections 2024-02-18T01:37:34.859 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- 172.21.15.105:0/3687633712 >> 172.21.15.105:0/3687633712 conn(0x7efd980f35d0 msgr2=0x7efd980f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:34.859 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- 172.21.15.105:0/3687633712 shutdown_connections 2024-02-18T01:37:34.859 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- 172.21.15.105:0/3687633712 wait complete. 2024-02-18T01:37:34.859 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 Processor -- start 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- start start 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efd98117310 con 0x7efd980fd970 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efd981184f0 con 0x7efd981042a0 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9ceb5700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7efd981196f0 con 0x7efd98100790 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9659c700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7efd980fd970 0x7efd980f9ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41452/0 (socket says 172.21.15.105:41452) 2024-02-18T01:37:34.860 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd9659c700 1 -- 172.21.15.105:0/103473902 learned_addr learned my addr 172.21.15.105:0/103473902 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3528687027 0 0) 0x7efd98117310 con 0x7efd980fd970 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd840035f0 con 0x7efd980fd970 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1993680009 0 0) 0x7efd981196f0 con 0x7efd98100790 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd98117310 con 0x7efd98100790 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2101669575 0 0) 0x7efd840035f0 con 0x7efd980fd970 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd981196f0 con 0x7efd980fd970 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7efd88002910 con 0x7efd980fd970 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 151934253 0 0) 0x7efd98117310 con 0x7efd98100790 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7efd840035f0 con 0x7efd98100790 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 598499785 0 0) 0x7efd981184f0 con 0x7efd981042a0 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd98117310 con 0x7efd981042a0 2024-02-18T01:37:34.862 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.856+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7efd800017c0 con 0x7efd98100790 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1716320552 0 0) 0x7efd981196f0 con 0x7efd980fd970 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 >> v1:172.21.15.105:6790/0 conn(0x7efd98100790 legacy=0x7efd980fa200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 >> v1:172.21.15.155:6789/0 conn(0x7efd981042a0 legacy=0x7efd981159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd9811a8f0 con 0x7efd980fd970 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd981198e0 con 0x7efd980fd970 2024-02-18T01:37:34.863 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7efd98119d30 con 0x7efd980fd970 2024-02-18T01:37:34.864 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7efd88002c60 con 0x7efd980fd970 2024-02-18T01:37:34.864 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7efd88005b80 con 0x7efd980fd970 2024-02-18T01:37:34.864 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd78004ee0 con 0x7efd980fd970 2024-02-18T01:37:34.865 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7efd88001620 con 0x7efd980fd970 2024-02-18T01:37:34.865 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.860+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(734..734 src has 1..734) v4 ==== 7598+0+0 (unknown 1302953512 0 0) 0x7efd88078ba0 con 0x7efd980fd970 2024-02-18T01:37:34.869 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:34.868+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7efd8804c1c0 con 0x7efd980fd970 2024-02-18T01:37:35.046 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.044+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12} v 0) v1 -- 0x7efd78005710 con 0x7efd980fd970 2024-02-18T01:37:35.683 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.679+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 10 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]=0 pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' created v735) v1 ==== 176+0+0 (unknown 3852755461 0 0) 0x7efd8804a630 con 0x7efd980fd970 2024-02-18T01:37:35.802 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12} v 0) v1 -- 0x7efd780058d0 con 0x7efd980fd970 2024-02-18T01:37:35.802 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd8f7fe700 1 -- 172.21.15.105:0/103473902 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]=0 pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' already exists v735) v1 ==== 183+0+0 (unknown 13940579 0 0) 0x7efd8804bdb0 con 0x7efd980fd970 2024-02-18T01:37:35.804 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 >> v1:172.21.15.105:6800/2148176012 conn(0x7efd840622c0 legacy=0x7efd84064780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:35.805 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 >> v1:172.21.15.105:6789/0 conn(0x7efd980fd970 legacy=0x7efd980f9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:35.805 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 shutdown_connections 2024-02-18T01:37:35.805 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 >> 172.21.15.105:0/103473902 conn(0x7efd980f35d0 msgr2=0x7efd98107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:35.805 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.799+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 shutdown_connections 2024-02-18T01:37:35.805 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.803+0000 7efd9ceb5700 1 -- 172.21.15.105:0/103473902 wait complete. 2024-02-18T01:37:35.806 INFO:tasks.workunit.client.0.smithi105.stderr:pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' already exists 2024-02-18T01:37:35.815 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool application enable 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 rados 2024-02-18T01:37:35.937 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.931+0000 7fcf77c6a700 1 -- 172.21.15.105:0/2016896457 >> v1:172.21.15.105:6789/0 conn(0x7fcf700f8940 legacy=0x7fcf700f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:35.937 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.931+0000 7fcf77c6a700 1 -- 172.21.15.105:0/2016896457 shutdown_connections 2024-02-18T01:37:35.937 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.931+0000 7fcf77c6a700 1 -- 172.21.15.105:0/2016896457 >> 172.21.15.105:0/2016896457 conn(0x7fcf700f24d0 msgr2=0x7fcf700f48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:35.937 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.931+0000 7fcf77c6a700 1 -- 172.21.15.105:0/2016896457 shutdown_connections 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.931+0000 7fcf77c6a700 1 -- 172.21.15.105:0/2016896457 wait complete. 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 Processor -- start 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- start start 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf701173f0 con 0x7fcf700f9430 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf701185d0 con 0x7fcf700f8940 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fcf701197d0 con 0x7fcf701042a0 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf75205700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7fcf700f9430 0x7fcf700ffe20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41462/0 (socket says 172.21.15.105:41462) 2024-02-18T01:37:35.938 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf75205700 1 -- 172.21.15.105:0/3633271853 learned_addr learned my addr 172.21.15.105:0/3633271853 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:37:35.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2569950055 0 0) 0x7fcf701173f0 con 0x7fcf700f9430 2024-02-18T01:37:35.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c0035f0 con 0x7fcf700f9430 2024-02-18T01:37:35.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154962492 0 0) 0x7fcf701197d0 con 0x7fcf701042a0 2024-02-18T01:37:35.940 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf701173f0 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311913865 0 0) 0x7fcf701185d0 con 0x7fcf700f8940 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf701197d0 con 0x7fcf700f8940 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 3726211262 0 0) 0x7fcf701173f0 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf701185d0 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fcf68001b40 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 4271427638 0 0) 0x7fcf5c0035f0 con 0x7fcf700f9430 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fcf701173f0 con 0x7fcf700f9430 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fcf64001900 con 0x7fcf700f9430 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1785780402 0 0) 0x7fcf701185d0 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 >> v1:172.21.15.155:6789/0 conn(0x7fcf700f8940 legacy=0x7fcf700ff710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 >> v1:172.21.15.105:6789/0 conn(0x7fcf700f9430 legacy=0x7fcf700ffe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf7011a9d0 con 0x7fcf701042a0 2024-02-18T01:37:35.941 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf701175e0 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70117a30 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fcf68002290 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fcf680035e0 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf54004ee0 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.935+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fcf68001df0 con 0x7fcf701042a0 2024-02-18T01:37:35.942 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.939+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(735..735 src has 1..735) v4 ==== 7973+0+0 (unknown 1952716720 0 0) 0x7fcf68076710 con 0x7fcf701042a0 2024-02-18T01:37:35.945 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:35.943+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fcf68049ba0 con 0x7fcf701042a0 2024-02-18T01:37:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: audit 2024-02-18T01:37:34.202382+0000 mon.a (mon.0) 3262 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: cluster 2024-02-18T01:37:34.202514+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T01:37:35.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: audit 2024-02-18T01:37:34.322955+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: audit 2024-02-18T01:37:34.323567+0000 mon.a (mon.0) 3264 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: cluster 2024-02-18T01:37:34.669536+0000 mon.a (mon.0) 3265 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: cluster 2024-02-18T01:37:34.669983+0000 mon.a (mon.0) 3266 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: audit 2024-02-18T01:37:34.679589+0000 mon.a (mon.0) 3267 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[27669]: cluster 2024-02-18T01:37:34.679693+0000 mon.a (mon.0) 3268 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: audit 2024-02-18T01:37:34.202382+0000 mon.a (mon.0) 3262 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: cluster 2024-02-18T01:37:34.202514+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: audit 2024-02-18T01:37:34.322955+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: audit 2024-02-18T01:37:34.323567+0000 mon.a (mon.0) 3264 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: cluster 2024-02-18T01:37:34.669536+0000 mon.a (mon.0) 3265 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:35.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: cluster 2024-02-18T01:37:34.669983+0000 mon.a (mon.0) 3266 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:35.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: audit 2024-02-18T01:37:34.679589+0000 mon.a (mon.0) 3267 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.991 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:35 smithi105 bash[19734]: cluster 2024-02-18T01:37:34.679693+0000 mon.a (mon.0) 3268 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: audit 2024-02-18T01:37:34.202382+0000 mon.a (mon.0) 3262 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: cluster 2024-02-18T01:37:34.202514+0000 mon.a (mon.0) 3263 : cluster [DBG] osdmap e733: 8 total, 8 up, 8 in 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: audit 2024-02-18T01:37:34.322955+0000 mon.c (mon.2) 422 : audit [INF] from='client.? v1:172.21.15.105:0/1906352619' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: audit 2024-02-18T01:37:34.323567+0000 mon.a (mon.0) 3264 : audit [INF] from='client.48422 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: cluster 2024-02-18T01:37:34.669536+0000 mon.a (mon.0) 3265 : cluster [INF] pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' no longer out of quota; removing NO_QUOTA flag 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: cluster 2024-02-18T01:37:34.669983+0000 mon.a (mon.0) 3266 : cluster [INF] Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: audit 2024-02-18T01:37:34.679589+0000 mon.a (mon.0) 3267 : audit [INF] from='client.48422 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:37:35.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:35 smithi155 bash[20805]: cluster 2024-02-18T01:37:34.679693+0000 mon.a (mon.0) 3268 : cluster [DBG] osdmap e734: 8 total, 8 up, 8 in 2024-02-18T01:37:36.108 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:36.103+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"} v 0) v1 -- 0x7fcf54005190 con 0x7fcf701042a0 2024-02-18T01:37:36.687 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:36.683+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]=0 enabled application 'rados' on pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' v736) v1 ==== 213+0+0 (unknown 2510905177 0 0) 0x7fcf68047fb0 con 0x7fcf701042a0 2024-02-18T01:37:36.806 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:36.803+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"} v 0) v1 -- 0x7fcf54002220 con 0x7fcf701042a0 2024-02-18T01:37:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:36] "GET /metrics HTTP/1.1" 200 191844 "" "Prometheus/2.33.4" 2024-02-18T01:37:36.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: audit 2024-02-18T01:37:35.046645+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: audit 2024-02-18T01:37:35.683041+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]': finished 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: cluster 2024-02-18T01:37:35.683272+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: audit 2024-02-18T01:37:35.802590+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: cluster 2024-02-18T01:37:36.075031+0000 mgr.y (mgr.24475) 787 : cluster [DBG] pgmap v1277: 188 pgs: 12 unknown, 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: audit 2024-02-18T01:37:36.109182+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[19734]: audit 2024-02-18T01:37:36.109830+0000 mon.a (mon.0) 3273 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: audit 2024-02-18T01:37:35.046645+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: audit 2024-02-18T01:37:35.683041+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]': finished 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: cluster 2024-02-18T01:37:35.683272+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T01:37:36.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: audit 2024-02-18T01:37:35.802590+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: cluster 2024-02-18T01:37:36.075031+0000 mgr.y (mgr.24475) 787 : cluster [DBG] pgmap v1277: 188 pgs: 12 unknown, 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:37:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: audit 2024-02-18T01:37:36.109182+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:36.991 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:36 smithi105 bash[27669]: audit 2024-02-18T01:37:36.109830+0000 mon.a (mon.0) 3273 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: audit 2024-02-18T01:37:35.046645+0000 mon.a (mon.0) 3269 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: audit 2024-02-18T01:37:35.683041+0000 mon.a (mon.0) 3270 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]': finished 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: cluster 2024-02-18T01:37:35.683272+0000 mon.a (mon.0) 3271 : cluster [DBG] osdmap e735: 8 total, 8 up, 8 in 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: audit 2024-02-18T01:37:35.802590+0000 mon.a (mon.0) 3272 : audit [INF] from='client.? v1:172.21.15.105:0/103473902' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pg_num": 12}]: dispatch 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: cluster 2024-02-18T01:37:36.075031+0000 mgr.y (mgr.24475) 787 : cluster [DBG] pgmap v1277: 188 pgs: 12 unknown, 176 active+clean; 143 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: audit 2024-02-18T01:37:36.109182+0000 mon.c (mon.2) 423 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:36.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:36 smithi155 bash[20805]: audit 2024-02-18T01:37:36.109830+0000 mon.a (mon.0) 3273 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.690 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.687+0000 7fcf62ffd700 1 -- 172.21.15.105:0/3633271853 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]=0 enabled application 'rados' on pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' v737) v1 ==== 213+0+0 (unknown 2944969151 0 0) 0x7fcf68049730 con 0x7fcf701042a0 2024-02-18T01:37:37.692 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.687+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 >> v1:172.21.15.105:6800/2148176012 conn(0x7fcf5c062190 legacy=0x7fcf5c064650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:37.692 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.687+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 >> v1:172.21.15.105:6790/0 conn(0x7fcf701042a0 legacy=0x7fcf70115ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:37.692 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.687+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 shutdown_connections 2024-02-18T01:37:37.693 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.687+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 >> 172.21.15.105:0/3633271853 conn(0x7fcf700f24d0 msgr2=0x7fcf70107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:37.693 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.691+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 shutdown_connections 2024-02-18T01:37:37.693 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.691+0000 7fcf77c6a700 1 -- 172.21.15.105:0/3633271853 wait complete. 2024-02-18T01:37:37.693 INFO:tasks.workunit.client.0.smithi105.stderr:enabled application 'rados' on pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' 2024-02-18T01:37:37.705 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 max_objects 10 2024-02-18T01:37:37.828 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.823+0000 7fac73f03700 1 -- 172.21.15.105:0/4286625029 >> v1:172.21.15.105:6790/0 conn(0x7fac6c104280 legacy=0x7fac6c106720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:37.828 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.823+0000 7fac73f03700 1 -- 172.21.15.105:0/4286625029 shutdown_connections 2024-02-18T01:37:37.828 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.823+0000 7fac73f03700 1 -- 172.21.15.105:0/4286625029 >> 172.21.15.105:0/4286625029 conn(0x7fac6c0f35d0 msgr2=0x7fac6c0f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:37.828 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.823+0000 7fac73f03700 1 -- 172.21.15.105:0/4286625029 shutdown_connections 2024-02-18T01:37:37.828 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.823+0000 7fac73f03700 1 -- 172.21.15.105:0/4286625029 wait complete. 2024-02-18T01:37:37.829 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 Processor -- start 2024-02-18T01:37:37.829 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- start start 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fac6c06cb50 con 0x7fac6c100770 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fac6c06cce0 con 0x7fac6c0fd950 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fac6c06ceb0 con 0x7fac6c104280 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac724a0700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fac6c104280 0x7fac6c1183e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:35276/0 (socket says 172.21.15.105:35276) 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac724a0700 1 -- 172.21.15.105:0/165685881 learned_addr learned my addr 172.21.15.105:0/165685881 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:37:37.830 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac7149e700 1 --1- 172.21.15.105:0/165685881 >> v1:172.21.15.105:6789/0 conn(0x7fac6c100770 0x7fac6c114cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:41476/0 (socket says 172.21.15.105:41476) 2024-02-18T01:37:37.832 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1518342828 0 0) 0x7fac6c06ceb0 con 0x7fac6c104280 2024-02-18T01:37:37.832 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac580033b0 con 0x7fac6c104280 2024-02-18T01:37:37.832 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3284759618 0 0) 0x7fac6c06cb50 con 0x7fac6c100770 2024-02-18T01:37:37.832 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac6c06ceb0 con 0x7fac6c100770 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1065633606 0 0) 0x7fac6c06cce0 con 0x7fac6c0fd950 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac6c06cb50 con 0x7fac6c0fd950 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1535420554 0 0) 0x7fac580033b0 con 0x7fac6c104280 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac6c06cce0 con 0x7fac6c104280 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fac64003090 con 0x7fac6c104280 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 958728365 0 0) 0x7fac6c06ceb0 con 0x7fac6c100770 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fac580033b0 con 0x7fac6c100770 2024-02-18T01:37:37.833 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fac60001880 con 0x7fac6c100770 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 851228595 0 0) 0x7fac6c06cce0 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 >> v1:172.21.15.155:6789/0 conn(0x7fac6c0fd950 legacy=0x7fac6c0fc8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 >> v1:172.21.15.105:6789/0 conn(0x7fac6c100770 legacy=0x7fac6c114cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac6c11cb30 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac6c119b60 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fac6c119fb0 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fac64001de0 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fac64003b80 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.827+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac50004ee0 con 0x7fac6c104280 2024-02-18T01:37:37.834 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.831+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fac64003ce0 con 0x7fac6c104280 2024-02-18T01:37:37.835 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.831+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(737..737 src has 1..737) v4 ==== 7986+0+0 (unknown 284366481 0 0) 0x7fac64079160 con 0x7fac6c104280 2024-02-18T01:37:37.838 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:37.835+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fac6404c5a0 con 0x7fac6c104280 2024-02-18T01:37:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[19734]: audit 2024-02-18T01:37:36.523489+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[19734]: audit 2024-02-18T01:37:36.684769+0000 mon.a (mon.0) 3275 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[19734]: cluster 2024-02-18T01:37:36.685331+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T01:37:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[19734]: audit 2024-02-18T01:37:36.806267+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[19734]: audit 2024-02-18T01:37:36.806736+0000 mon.a (mon.0) 3277 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[27669]: audit 2024-02-18T01:37:36.523489+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[27669]: audit 2024-02-18T01:37:36.684769+0000 mon.a (mon.0) 3275 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[27669]: cluster 2024-02-18T01:37:36.685331+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T01:37:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[27669]: audit 2024-02-18T01:37:36.806267+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:37 smithi105 bash[27669]: audit 2024-02-18T01:37:36.806736+0000 mon.a (mon.0) 3277 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:37 smithi155 bash[20805]: audit 2024-02-18T01:37:36.523489+0000 mon.a (mon.0) 3274 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:37:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:37 smithi155 bash[20805]: audit 2024-02-18T01:37:36.684769+0000 mon.a (mon.0) 3275 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:37.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:37 smithi155 bash[20805]: cluster 2024-02-18T01:37:36.685331+0000 mon.a (mon.0) 3276 : cluster [DBG] osdmap e736: 8 total, 8 up, 8 in 2024-02-18T01:37:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:37 smithi155 bash[20805]: audit 2024-02-18T01:37:36.806267+0000 mon.c (mon.2) 424 : audit [INF] from='client.? v1:172.21.15.105:0/3633271853' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:37.994 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:37 smithi155 bash[20805]: audit 2024-02-18T01:37:36.806736+0000 mon.a (mon.0) 3277 : audit [INF] from='client.48443 ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]: dispatch 2024-02-18T01:37:38.017 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:38.015+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fac50005190 con 0x7fac6c104280 2024-02-18T01:37:38.692 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:38.692 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[20805]: audit 2024-02-18T01:37:37.687790+0000 mon.a (mon.0) 3278 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:38.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[20805]: cluster 2024-02-18T01:37:37.688300+0000 mon.a (mon.0) 3279 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T01:37:38.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[20805]: audit 2024-02-18T01:37:38.017790+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[20805]: audit 2024-02-18T01:37:38.018324+0000 mon.a (mon.0) 3280 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.693 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:38 smithi155 bash[20805]: cluster 2024-02-18T01:37:38.075921+0000 mgr.y (mgr.24475) 788 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-18T01:37:38.695 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:38.691+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 v738) v1 ==== 223+0+0 (unknown 1718756665 0 0) 0x7fac6404aa10 con 0x7fac6c104280 2024-02-18T01:37:38.813 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:38.807+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fac50005800 con 0x7fac6c104280 2024-02-18T01:37:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[27669]: audit 2024-02-18T01:37:37.687790+0000 mon.a (mon.0) 3278 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[27669]: cluster 2024-02-18T01:37:37.688300+0000 mon.a (mon.0) 3279 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T01:37:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[27669]: audit 2024-02-18T01:37:38.017790+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[27669]: audit 2024-02-18T01:37:38.018324+0000 mon.a (mon.0) 3280 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[27669]: cluster 2024-02-18T01:37:38.075921+0000 mgr.y (mgr.24475) 788 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-18T01:37:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[19734]: audit 2024-02-18T01:37:37.687790+0000 mon.a (mon.0) 3278 : audit [INF] from='client.48443 ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "app": "rados"}]': finished 2024-02-18T01:37:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[19734]: cluster 2024-02-18T01:37:37.688300+0000 mon.a (mon.0) 3279 : cluster [DBG] osdmap e737: 8 total, 8 up, 8 in 2024-02-18T01:37:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[19734]: audit 2024-02-18T01:37:38.017790+0000 mon.c (mon.2) 425 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[19734]: audit 2024-02-18T01:37:38.018324+0000 mon.a (mon.0) 3280 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:38.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:38 smithi105 bash[19734]: cluster 2024-02-18T01:37:38.075921+0000 mgr.y (mgr.24475) 788 : cluster [DBG] pgmap v1280: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-02-18T01:37:39.705 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac5effd700 1 -- 172.21.15.105:0/165685881 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 v739) v1 ==== 223+0+0 (unknown 792166473 0 0) 0x7fac6404c190 con 0x7fac6c104280 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 >> v1:172.21.15.105:6800/2148176012 conn(0x7fac58062200 legacy=0x7fac580646c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 >> v1:172.21.15.105:6790/0 conn(0x7fac6c104280 legacy=0x7fac6c1183e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 shutdown_connections 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 >> 172.21.15.105:0/165685881 conn(0x7fac6c0f35d0 msgr2=0x7fac6c107710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 shutdown_connections 2024-02-18T01:37:39.708 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:37:39.703+0000 7fac73f03700 1 -- 172.21.15.105:0/165685881 wait complete. 2024-02-18T01:37:39.709 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_objects = 10 for pool 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 2024-02-18T01:37:39.720 INFO:tasks.workunit.client.0.smithi105.stderr:+ sleep 30 2024-02-18T01:37:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[19734]: audit 2024-02-18T01:37:38.336800+0000 mgr.y (mgr.24475) 789 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[19734]: audit 2024-02-18T01:37:38.694259+0000 mon.a (mon.0) 3281 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[19734]: cluster 2024-02-18T01:37:38.694394+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T01:37:39.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[19734]: audit 2024-02-18T01:37:38.813555+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[19734]: audit 2024-02-18T01:37:38.814174+0000 mon.a (mon.0) 3283 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[27669]: audit 2024-02-18T01:37:38.336800+0000 mgr.y (mgr.24475) 789 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[27669]: audit 2024-02-18T01:37:38.694259+0000 mon.a (mon.0) 3281 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[27669]: cluster 2024-02-18T01:37:38.694394+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[27669]: audit 2024-02-18T01:37:38.813555+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:39.990 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:39 smithi105 bash[27669]: audit 2024-02-18T01:37:38.814174+0000 mon.a (mon.0) 3283 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:39 smithi155 bash[20805]: audit 2024-02-18T01:37:38.336800+0000 mgr.y (mgr.24475) 789 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:39 smithi155 bash[20805]: audit 2024-02-18T01:37:38.694259+0000 mon.a (mon.0) 3281 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:39 smithi155 bash[20805]: cluster 2024-02-18T01:37:38.694394+0000 mon.a (mon.0) 3282 : cluster [DBG] osdmap e738: 8 total, 8 up, 8 in 2024-02-18T01:37:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:39 smithi155 bash[20805]: audit 2024-02-18T01:37:38.813555+0000 mon.c (mon.2) 426 : audit [INF] from='client.? v1:172.21.15.105:0/165685881' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:39.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:39 smithi155 bash[20805]: audit 2024-02-18T01:37:38.814174+0000 mon.a (mon.0) 3283 : audit [INF] from='client.48449 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]: dispatch 2024-02-18T01:37:40.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[27669]: audit 2024-02-18T01:37:39.704516+0000 mon.a (mon.0) 3284 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:40.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[27669]: cluster 2024-02-18T01:37:39.704649+0000 mon.a (mon.0) 3285 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T01:37:40.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[27669]: cluster 2024-02-18T01:37:40.076760+0000 mgr.y (mgr.24475) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-18T01:37:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[19734]: audit 2024-02-18T01:37:39.704516+0000 mon.a (mon.0) 3284 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[19734]: cluster 2024-02-18T01:37:39.704649+0000 mon.a (mon.0) 3285 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T01:37:40.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:40 smithi105 bash[19734]: cluster 2024-02-18T01:37:40.076760+0000 mgr.y (mgr.24475) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-18T01:37:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:40 smithi155 bash[20805]: audit 2024-02-18T01:37:39.704516+0000 mon.a (mon.0) 3284 : audit [INF] from='client.48449 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "field": "max_objects", "val": "10"}]': finished 2024-02-18T01:37:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:40 smithi155 bash[20805]: cluster 2024-02-18T01:37:39.704649+0000 mon.a (mon.0) 3285 : cluster [DBG] osdmap e739: 8 total, 8 up, 8 in 2024-02-18T01:37:40.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:40 smithi155 bash[20805]: cluster 2024-02-18T01:37:40.076760+0000 mgr.y (mgr.24475) 790 : cluster [DBG] pgmap v1283: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-02-18T01:37:42.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:43.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:43 smithi105 bash[19734]: cluster 2024-02-18T01:37:42.077624+0000 mgr.y (mgr.24475) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-18T01:37:43.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:43 smithi105 bash[27669]: cluster 2024-02-18T01:37:42.077624+0000 mgr.y (mgr.24475) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-18T01:37:43.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:43 smithi155 bash[20805]: cluster 2024-02-18T01:37:42.077624+0000 mgr.y (mgr.24475) 791 : cluster [DBG] pgmap v1284: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 948 B/s rd, 4.4 KiB/s wr, 1 op/s 2024-02-18T01:37:45.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:45 smithi105 bash[19734]: cluster 2024-02-18T01:37:44.079504+0000 mgr.y (mgr.24475) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-02-18T01:37:45.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:45 smithi105 bash[27669]: cluster 2024-02-18T01:37:44.079504+0000 mgr.y (mgr.24475) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-02-18T01:37:45.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:45 smithi155 bash[20805]: cluster 2024-02-18T01:37:44.079504+0000 mgr.y (mgr.24475) 792 : cluster [DBG] pgmap v1285: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 800 B/s rd, 0 op/s 2024-02-18T01:37:46.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:46] "GET /metrics HTTP/1.1" 200 191844 "" "Prometheus/2.33.4" 2024-02-18T01:37:47.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:47 smithi105 bash[27669]: cluster 2024-02-18T01:37:46.080291+0000 mgr.y (mgr.24475) 793 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:37:47.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:47 smithi105 bash[19734]: cluster 2024-02-18T01:37:46.080291+0000 mgr.y (mgr.24475) 793 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:37:47.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:47 smithi155 bash[20805]: cluster 2024-02-18T01:37:46.080291+0000 mgr.y (mgr.24475) 793 : cluster [DBG] pgmap v1286: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-02-18T01:37:48.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:49 smithi105 bash[19734]: cluster 2024-02-18T01:37:48.082352+0000 mgr.y (mgr.24475) 794 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:37:49.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:49 smithi105 bash[27669]: cluster 2024-02-18T01:37:48.082352+0000 mgr.y (mgr.24475) 794 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:37:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:49 smithi155 bash[20805]: cluster 2024-02-18T01:37:48.082352+0000 mgr.y (mgr.24475) 794 : cluster [DBG] pgmap v1287: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-02-18T01:37:50.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:50 smithi105 bash[27669]: audit 2024-02-18T01:37:48.345623+0000 mgr.y (mgr.24475) 795 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:50 smithi105 bash[19734]: audit 2024-02-18T01:37:48.345623+0000 mgr.y (mgr.24475) 795 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:50.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:50 smithi155 bash[20805]: audit 2024-02-18T01:37:48.345623+0000 mgr.y (mgr.24475) 795 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:37:51.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:51 smithi105 bash[27669]: cluster 2024-02-18T01:37:50.083140+0000 mgr.y (mgr.24475) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s 2024-02-18T01:37:51.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:51 smithi105 bash[19734]: cluster 2024-02-18T01:37:50.083140+0000 mgr.y (mgr.24475) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s 2024-02-18T01:37:51.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:51 smithi155 bash[20805]: cluster 2024-02-18T01:37:50.083140+0000 mgr.y (mgr.24475) 796 : cluster [DBG] pgmap v1288: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 986 B/s rd, 0 op/s 2024-02-18T01:37:52.166 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:37:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:37:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:52 smithi105 bash[27669]: cluster 2024-02-18T01:37:52.083944+0000 mgr.y (mgr.24475) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:52.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:52 smithi105 bash[19734]: cluster 2024-02-18T01:37:52.083944+0000 mgr.y (mgr.24475) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:52.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:52 smithi155 bash[20805]: cluster 2024-02-18T01:37:52.083944+0000 mgr.y (mgr.24475) 797 : cluster [DBG] pgmap v1289: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:55.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:55 smithi105 bash[19734]: cluster 2024-02-18T01:37:54.085892+0000 mgr.y (mgr.24475) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:55.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:55 smithi105 bash[27669]: cluster 2024-02-18T01:37:54.085892+0000 mgr.y (mgr.24475) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:55.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:55 smithi155 bash[20805]: cluster 2024-02-18T01:37:54.085892+0000 mgr.y (mgr.24475) 798 : cluster [DBG] pgmap v1290: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:37:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:37:56] "GET /metrics HTTP/1.1" 200 191860 "" "Prometheus/2.33.4" 2024-02-18T01:37:57.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:57 smithi105 bash[27669]: cluster 2024-02-18T01:37:56.086664+0000 mgr.y (mgr.24475) 799 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:57.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:57 smithi105 bash[19734]: cluster 2024-02-18T01:37:56.086664+0000 mgr.y (mgr.24475) 799 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:57.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:57 smithi155 bash[20805]: cluster 2024-02-18T01:37:56.086664+0000 mgr.y (mgr.24475) 799 : cluster [DBG] pgmap v1291: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:37:58.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:37:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:37:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:37:59 smithi105 bash[19734]: cluster 2024-02-18T01:37:58.088619+0000 mgr.y (mgr.24475) 800 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:59.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:37:59 smithi105 bash[27669]: cluster 2024-02-18T01:37:58.088619+0000 mgr.y (mgr.24475) 800 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:37:59.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:37:59 smithi155 bash[20805]: cluster 2024-02-18T01:37:58.088619+0000 mgr.y (mgr.24475) 800 : cluster [DBG] pgmap v1292: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:00 smithi105 bash[19734]: audit 2024-02-18T01:37:58.356839+0000 mgr.y (mgr.24475) 801 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:00 smithi105 bash[19734]: cluster 2024-02-18T01:38:00.089425+0000 mgr.y (mgr.24475) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:00.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:00 smithi105 bash[27669]: audit 2024-02-18T01:37:58.356839+0000 mgr.y (mgr.24475) 801 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:00.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:00 smithi105 bash[27669]: cluster 2024-02-18T01:38:00.089425+0000 mgr.y (mgr.24475) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:00 smithi155 bash[20805]: audit 2024-02-18T01:37:58.356839+0000 mgr.y (mgr.24475) 801 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:00 smithi155 bash[20805]: cluster 2024-02-18T01:38:00.089425+0000 mgr.y (mgr.24475) 802 : cluster [DBG] pgmap v1293: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:02.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:03.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:03 smithi105 bash[19734]: cluster 2024-02-18T01:38:02.090346+0000 mgr.y (mgr.24475) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:03.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:03 smithi105 bash[27669]: cluster 2024-02-18T01:38:02.090346+0000 mgr.y (mgr.24475) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:03.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:03 smithi155 bash[20805]: cluster 2024-02-18T01:38:02.090346+0000 mgr.y (mgr.24475) 803 : cluster [DBG] pgmap v1294: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:05.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:05 smithi105 bash[19734]: cluster 2024-02-18T01:38:04.092361+0000 mgr.y (mgr.24475) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:05.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:05 smithi105 bash[27669]: cluster 2024-02-18T01:38:04.092361+0000 mgr.y (mgr.24475) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:05.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:05 smithi155 bash[20805]: cluster 2024-02-18T01:38:04.092361+0000 mgr.y (mgr.24475) 804 : cluster [DBG] pgmap v1295: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:06.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:06 smithi105 bash[19734]: cluster 2024-02-18T01:38:06.093142+0000 mgr.y (mgr.24475) 805 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:06.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:06 smithi105 bash[27669]: cluster 2024-02-18T01:38:06.093142+0000 mgr.y (mgr.24475) 805 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:06.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:06 smithi155 bash[20805]: cluster 2024-02-18T01:38:06.093142+0000 mgr.y (mgr.24475) 805 : cluster [DBG] pgmap v1296: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:06] "GET /metrics HTTP/1.1" 200 191861 "" "Prometheus/2.33.4" 2024-02-18T01:38:08.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:09.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:09 smithi105 bash[27669]: cluster 2024-02-18T01:38:08.095098+0000 mgr.y (mgr.24475) 806 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:09 smithi105 bash[19734]: cluster 2024-02-18T01:38:08.095098+0000 mgr.y (mgr.24475) 806 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:09 smithi155 bash[20805]: cluster 2024-02-18T01:38:08.095098+0000 mgr.y (mgr.24475) 806 : cluster [DBG] pgmap v1297: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:09.722 INFO:tasks.workunit.client.0.smithi105.stderr:+ seq 1 10 2024-02-18T01:38:09.722 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj1 /etc/passwd 2024-02-18T01:38:09.771 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj2 /etc/passwd 2024-02-18T01:38:09.812 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj3 /etc/passwd 2024-02-18T01:38:09.855 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj4 /etc/passwd 2024-02-18T01:38:09.898 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj5 /etc/passwd 2024-02-18T01:38:09.939 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj6 /etc/passwd 2024-02-18T01:38:09.984 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj7 /etc/passwd 2024-02-18T01:38:10.027 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj8 /etc/passwd 2024-02-18T01:38:10.068 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj9 /etc/passwd 2024-02-18T01:38:10.108 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 put obj10 /etc/passwd 2024-02-18T01:38:10.148 INFO:tasks.workunit.client.0.smithi105.stderr:+ sleep 30 2024-02-18T01:38:10.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:10 smithi105 bash[19734]: audit 2024-02-18T01:38:08.361819+0000 mgr.y (mgr.24475) 807 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:10 smithi105 bash[27669]: audit 2024-02-18T01:38:08.361819+0000 mgr.y (mgr.24475) 807 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:10 smithi155 bash[20805]: audit 2024-02-18T01:38:08.361819+0000 mgr.y (mgr.24475) 807 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:11.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:11 smithi105 bash[19734]: cluster 2024-02-18T01:38:10.095874+0000 mgr.y (mgr.24475) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:11.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:11 smithi105 bash[27669]: cluster 2024-02-18T01:38:10.095874+0000 mgr.y (mgr.24475) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:11.492 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:11 smithi155 bash[20805]: cluster 2024-02-18T01:38:10.095874+0000 mgr.y (mgr.24475) 808 : cluster [DBG] pgmap v1298: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:12.174 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:12.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:12 smithi105 bash[19734]: cluster 2024-02-18T01:38:12.096693+0000 mgr.y (mgr.24475) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:12.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:12 smithi105 bash[27669]: cluster 2024-02-18T01:38:12.096693+0000 mgr.y (mgr.24475) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:12 smithi155 bash[20805]: cluster 2024-02-18T01:38:12.096693+0000 mgr.y (mgr.24475) 809 : cluster [DBG] pgmap v1299: 188 pgs: 188 active+clean; 166 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[19734]: cluster 2024-02-18T01:38:14.098728+0000 mgr.y (mgr.24475) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-18T01:38:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[19734]: cluster 2024-02-18T01:38:14.695881+0000 mon.a (mon.0) 3286 : cluster [WRN] pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' is full (reached quota's max_objects: 10) 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[19734]: cluster 2024-02-18T01:38:14.696351+0000 mon.a (mon.0) 3287 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[19734]: cluster 2024-02-18T01:38:14.705720+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[27669]: cluster 2024-02-18T01:38:14.098728+0000 mgr.y (mgr.24475) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[27669]: cluster 2024-02-18T01:38:14.695881+0000 mon.a (mon.0) 3286 : cluster [WRN] pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' is full (reached quota's max_objects: 10) 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[27669]: cluster 2024-02-18T01:38:14.696351+0000 mon.a (mon.0) 3287 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:38:15.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:15 smithi105 bash[27669]: cluster 2024-02-18T01:38:14.705720+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T01:38:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:15 smithi155 bash[20805]: cluster 2024-02-18T01:38:14.098728+0000 mgr.y (mgr.24475) 810 : cluster [DBG] pgmap v1300: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-02-18T01:38:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:15 smithi155 bash[20805]: cluster 2024-02-18T01:38:14.695881+0000 mon.a (mon.0) 3286 : cluster [WRN] pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' is full (reached quota's max_objects: 10) 2024-02-18T01:38:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:15 smithi155 bash[20805]: cluster 2024-02-18T01:38:14.696351+0000 mon.a (mon.0) 3287 : cluster [WRN] Health check failed: 1 pool(s) full (POOL_FULL) 2024-02-18T01:38:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:15 smithi155 bash[20805]: cluster 2024-02-18T01:38:14.705720+0000 mon.a (mon.0) 3288 : cluster [DBG] osdmap e740: 8 total, 8 up, 8 in 2024-02-18T01:38:16.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:16 smithi105 bash[19734]: cluster 2024-02-18T01:38:16.099557+0000 mgr.y (mgr.24475) 811 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:16.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:16 smithi105 bash[27669]: cluster 2024-02-18T01:38:16.099557+0000 mgr.y (mgr.24475) 811 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:16 smithi155 bash[20805]: cluster 2024-02-18T01:38:16.099557+0000 mgr.y (mgr.24475) 811 : cluster [DBG] pgmap v1302: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:16.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:16] "GET /metrics HTTP/1.1" 200 191861 "" "Prometheus/2.33.4" 2024-02-18T01:38:18.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:18 smithi105 bash[19734]: cluster 2024-02-18T01:38:18.101459+0000 mgr.y (mgr.24475) 812 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:18.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:18 smithi105 bash[27669]: cluster 2024-02-18T01:38:18.101459+0000 mgr.y (mgr.24475) 812 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:18.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:18 smithi155 bash[20805]: cluster 2024-02-18T01:38:18.101459+0000 mgr.y (mgr.24475) 812 : cluster [DBG] pgmap v1303: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:18.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:19.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:19 smithi105 bash[27669]: audit 2024-02-18T01:38:18.368230+0000 mgr.y (mgr.24475) 813 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:19.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:19 smithi105 bash[19734]: audit 2024-02-18T01:38:18.368230+0000 mgr.y (mgr.24475) 813 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:19.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:19 smithi155 bash[20805]: audit 2024-02-18T01:38:18.368230+0000 mgr.y (mgr.24475) 813 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:20.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:20 smithi105 bash[27669]: cluster 2024-02-18T01:38:20.102274+0000 mgr.y (mgr.24475) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:20.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:20 smithi105 bash[19734]: cluster 2024-02-18T01:38:20.102274+0000 mgr.y (mgr.24475) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:20.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:20 smithi155 bash[20805]: cluster 2024-02-18T01:38:20.102274+0000 mgr.y (mgr.24475) 814 : cluster [DBG] pgmap v1304: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:22.243 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:22.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:22 smithi105 bash[27669]: audit 2024-02-18T01:38:21.527435+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:22.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:22 smithi105 bash[27669]: cluster 2024-02-18T01:38:22.103218+0000 mgr.y (mgr.24475) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:22.989 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:22 smithi105 bash[19734]: audit 2024-02-18T01:38:21.527435+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:22.990 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:22 smithi105 bash[19734]: cluster 2024-02-18T01:38:22.103218+0000 mgr.y (mgr.24475) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:22.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:22 smithi155 bash[20805]: audit 2024-02-18T01:38:21.527435+0000 mon.a (mon.0) 3289 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:22.993 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:22 smithi155 bash[20805]: cluster 2024-02-18T01:38:22.103218+0000 mgr.y (mgr.24475) 815 : cluster [DBG] pgmap v1305: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-02-18T01:38:24.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:24 smithi105 bash[19734]: cluster 2024-02-18T01:38:24.105438+0000 mgr.y (mgr.24475) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:24.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:24 smithi105 bash[27669]: cluster 2024-02-18T01:38:24.105438+0000 mgr.y (mgr.24475) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:24.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:24 smithi155 bash[20805]: cluster 2024-02-18T01:38:24.105438+0000 mgr.y (mgr.24475) 816 : cluster [DBG] pgmap v1306: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:26.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:26 smithi105 bash[27669]: cluster 2024-02-18T01:38:26.106259+0000 mgr.y (mgr.24475) 817 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-02-18T01:38:26.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:26 smithi105 bash[19734]: cluster 2024-02-18T01:38:26.106259+0000 mgr.y (mgr.24475) 817 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-02-18T01:38:26.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:26 smithi155 bash[20805]: cluster 2024-02-18T01:38:26.106259+0000 mgr.y (mgr.24475) 817 : cluster [DBG] pgmap v1307: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 897 B/s rd, 0 op/s 2024-02-18T01:38:26.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:26 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:26] "GET /metrics HTTP/1.1" 200 191889 "" "Prometheus/2.33.4" 2024-02-18T01:38:28.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:28 smithi105 bash[19734]: cluster 2024-02-18T01:38:28.108225+0000 mgr.y (mgr.24475) 818 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:28.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:28 smithi105 bash[27669]: cluster 2024-02-18T01:38:28.108225+0000 mgr.y (mgr.24475) 818 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:28.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:28 smithi155 bash[20805]: cluster 2024-02-18T01:38:28.108225+0000 mgr.y (mgr.24475) 818 : cluster [DBG] pgmap v1308: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:28.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:28 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[19734]: audit 2024-02-18T01:38:28.369823+0000 mgr.y (mgr.24475) 819 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[19734]: audit 2024-02-18T01:38:28.426107+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:38:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[19734]: audit 2024-02-18T01:38:28.749733+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:38:29.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[19734]: audit 2024-02-18T01:38:28.751362+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[19734]: audit 2024-02-18T01:38:28.758066+0000 mon.a (mon.0) 3293 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[27669]: audit 2024-02-18T01:38:28.369823+0000 mgr.y (mgr.24475) 819 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[27669]: audit 2024-02-18T01:38:28.426107+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[27669]: audit 2024-02-18T01:38:28.749733+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[27669]: audit 2024-02-18T01:38:28.751362+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:38:29.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:29 smithi105 bash[27669]: audit 2024-02-18T01:38:28.758066+0000 mon.a (mon.0) 3293 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:29 smithi155 bash[20805]: audit 2024-02-18T01:38:28.369823+0000 mgr.y (mgr.24475) 819 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:29 smithi155 bash[20805]: audit 2024-02-18T01:38:28.426107+0000 mon.a (mon.0) 3290 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-02-18T01:38:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:29 smithi155 bash[20805]: audit 2024-02-18T01:38:28.749733+0000 mon.a (mon.0) 3291 : audit [DBG] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-02-18T01:38:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:29 smithi155 bash[20805]: audit 2024-02-18T01:38:28.751362+0000 mon.a (mon.0) 3292 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-02-18T01:38:29.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:29 smithi155 bash[20805]: audit 2024-02-18T01:38:28.758066+0000 mon.a (mon.0) 3293 : audit [INF] from='mgr.24475 v1:172.21.15.105:0/3557159640' entity='mgr.y' 2024-02-18T01:38:30.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:30 smithi105 bash[19734]: cluster 2024-02-18T01:38:30.109080+0000 mgr.y (mgr.24475) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:30.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:30 smithi105 bash[27669]: cluster 2024-02-18T01:38:30.109080+0000 mgr.y (mgr.24475) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:30.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:30 smithi155 bash[20805]: cluster 2024-02-18T01:38:30.109080+0000 mgr.y (mgr.24475) 820 : cluster [DBG] pgmap v1309: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:32.213 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:31 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:31] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:32.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:32 smithi105 bash[19734]: cluster 2024-02-18T01:38:32.109896+0000 mgr.y (mgr.24475) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:32.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:32 smithi105 bash[27669]: cluster 2024-02-18T01:38:32.109896+0000 mgr.y (mgr.24475) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:32.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:32 smithi155 bash[20805]: cluster 2024-02-18T01:38:32.109896+0000 mgr.y (mgr.24475) 821 : cluster [DBG] pgmap v1310: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:34.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:34 smithi105 bash[19734]: cluster 2024-02-18T01:38:34.111885+0000 mgr.y (mgr.24475) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:34.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:34 smithi105 bash[27669]: cluster 2024-02-18T01:38:34.111885+0000 mgr.y (mgr.24475) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:34.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:34 smithi155 bash[20805]: cluster 2024-02-18T01:38:34.111885+0000 mgr.y (mgr.24475) 822 : cluster [DBG] pgmap v1311: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:36.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:36 smithi105 bash[19734]: cluster 2024-02-18T01:38:36.112707+0000 mgr.y (mgr.24475) 823 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:36.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:36 smithi105 bash[27669]: cluster 2024-02-18T01:38:36.112707+0000 mgr.y (mgr.24475) 823 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:36.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:36 smithi155 bash[20805]: cluster 2024-02-18T01:38:36.112707+0000 mgr.y (mgr.24475) 823 : cluster [DBG] pgmap v1312: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:36.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:36 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:36] "GET /metrics HTTP/1.1" 200 191897 "" "Prometheus/2.33.4" 2024-02-18T01:38:38.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:38 smithi105 bash[19734]: cluster 2024-02-18T01:38:38.114636+0000 mgr.y (mgr.24475) 824 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:38.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:38 smithi105 bash[27669]: cluster 2024-02-18T01:38:38.114636+0000 mgr.y (mgr.24475) 824 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:38.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:38 smithi155 bash[20805]: cluster 2024-02-18T01:38:38.114636+0000 mgr.y (mgr.24475) 824 : cluster [DBG] pgmap v1313: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:38.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:38 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:39.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:39 smithi105 bash[19734]: audit 2024-02-18T01:38:38.380645+0000 mgr.y (mgr.24475) 825 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:39.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:39 smithi105 bash[27669]: audit 2024-02-18T01:38:38.380645+0000 mgr.y (mgr.24475) 825 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:39.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:39 smithi155 bash[20805]: audit 2024-02-18T01:38:38.380645+0000 mgr.y (mgr.24475) 825 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:40.149 INFO:tasks.workunit.client.0.smithi105.stderr:+ rados -p 6cfab109-eaaa-440b-98e0-f8450a870fb2 put threemore /etc/passwd 2024-02-18T01:38:40.193 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_bytes 0 2024-02-18T01:38:40.317 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- 172.21.15.105:0/2440223847 >> v1:172.21.15.105:6790/0 conn(0x7f26ec06bba0 legacy=0x7f26ec104bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:40.317 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- 172.21.15.105:0/2440223847 shutdown_connections 2024-02-18T01:38:40.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- 172.21.15.105:0/2440223847 >> 172.21.15.105:0/2440223847 conn(0x7f26ec0f3450 msgr2=0x7f26ec0f5870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:38:40.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- 172.21.15.105:0/2440223847 shutdown_connections 2024-02-18T01:38:40.318 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- 172.21.15.105:0/2440223847 wait complete. 2024-02-18T01:38:40.319 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 Processor -- start 2024-02-18T01:38:40.319 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- start start 2024-02-18T01:38:40.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f26ec119790 con 0x7f26ec106360 2024-02-18T01:38:40.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f26ec11a930 con 0x7f26ec06bba0 2024-02-18T01:38:40.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26f0dbd700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f26ec11bad0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.320 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26eb7fe700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7f26ec06b0b0 0x7f26ec1054f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:46660/0 (socket says 172.21.15.105:46660) 2024-02-18T01:38:40.321 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26eb7fe700 1 -- 172.21.15.105:0/1876090042 learned_addr learned my addr 172.21.15.105:0/1876090042 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:38:40.321 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1177825408 0 0) 0x7f26ec11bad0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.321 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d80035f0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1217254792 0 0) 0x7f26ec119790 con 0x7f26ec106360 2024-02-18T01:38:40.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26ec11bad0 con 0x7f26ec106360 2024-02-18T01:38:40.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.314+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2925698453 0 0) 0x7f26d80035f0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.322 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f26ec119790 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f26d4001940 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 944872270 0 0) 0x7f26ec11a930 con 0x7f26ec06bba0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26d80035f0 con 0x7f26ec06bba0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 602957218 0 0) 0x7f26ec119790 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 >> v1:172.21.15.155:6789/0 conn(0x7f26ec06bba0 legacy=0x7f26ec114940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 >> v1:172.21.15.105:6789/0 conn(0x7f26ec106360 legacy=0x7f26ec118070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26ec11cc70 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26ec119980 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f26d4001c00 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f26d4005b80 con 0x7f26ec06b0b0 2024-02-18T01:38:40.323 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f26ec119dd0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ec104110 con 0x7f26ec06b0b0 2024-02-18T01:38:40.324 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f26d4006e40 con 0x7f26ec06b0b0 2024-02-18T01:38:40.325 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.318+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(740..740 src has 1..740) v4 ==== 7986+0+0 (unknown 4097577041 0 0) 0x7f26d40791b0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.325 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.322+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=741}) v3 -- 0x7f26ec119790 con 0x7f26ec06b0b0 2024-02-18T01:38:40.329 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.326+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f26d404d0a0 con 0x7f26ec06b0b0 2024-02-18T01:38:40.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:40 smithi105 bash[27669]: cluster 2024-02-18T01:38:40.115464+0000 mgr.y (mgr.24475) 826 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:40.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:40 smithi105 bash[19734]: cluster 2024-02-18T01:38:40.115464+0000 mgr.y (mgr.24475) 826 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:40.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:40 smithi155 bash[20805]: cluster 2024-02-18T01:38:40.115464+0000 mgr.y (mgr.24475) 826 : cluster [DBG] pgmap v1314: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:38:40.508 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:40.502+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f26ec05f240 con 0x7f26ec06b0b0 2024-02-18T01:38:41.238 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:41.234+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 10 ==== osd_map(741..741 src has 1..741) v4 ==== 620+0+0 (unknown 92340233 0 0) 0x7f26d404d2d0 con 0x7f26ec06b0b0 2024-02-18T01:38:41.238 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:41.234+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=742}) v3 -- 0x7f26ec11a930 con 0x7f26ec06b0b0 2024-02-18T01:38:41.245 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:41.238+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v741) v1 ==== 217+0+0 (unknown 2074861552 0 0) 0x7f26d404b4b0 con 0x7f26ec06b0b0 2024-02-18T01:38:41.360 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:41.354+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f26ec046ca0 con 0x7f26ec06b0b0 2024-02-18T01:38:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:41 smithi105 bash[19734]: audit 2024-02-18T01:38:40.508679+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:41.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:41 smithi105 bash[19734]: audit 2024-02-18T01:38:40.509312+0000 mon.a (mon.0) 3294 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:41 smithi105 bash[27669]: audit 2024-02-18T01:38:40.508679+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:41.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:41 smithi105 bash[27669]: audit 2024-02-18T01:38:40.509312+0000 mon.a (mon.0) 3294 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:41 smithi155 bash[20805]: audit 2024-02-18T01:38:40.508679+0000 mon.c (mon.2) 427 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:41.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:41 smithi155 bash[20805]: audit 2024-02-18T01:38:40.509312+0000 mon.a (mon.0) 3294 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.239 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:41 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:41] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:42.251 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.246+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 12 ==== osd_map(742..742 src has 1..742) v4 ==== 620+0+0 (unknown 2868888660 0 0) 0x7f26d404b670 con 0x7f26ec06b0b0 2024-02-18T01:38:42.251 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.246+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=743}) v3 -- 0x7f26d806f890 con 0x7f26ec06b0b0 2024-02-18T01:38:42.254 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.250+0000 7f26e8ff9700 1 -- 172.21.15.105:0/1876090042 <== mon.2 v1:172.21.15.105:6790/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v742) v1 ==== 217+0+0 (unknown 229293907 0 0) 0x7f26d404cc30 con 0x7f26ec06b0b0 2024-02-18T01:38:42.258 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 >> v1:172.21.15.105:6800/2148176012 conn(0x7f26d8062170 legacy=0x7f26d8064630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:42.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 >> v1:172.21.15.105:6790/0 conn(0x7f26ec06b0b0 legacy=0x7f26ec1054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:42.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 shutdown_connections 2024-02-18T01:38:42.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 >> 172.21.15.105:0/1876090042 conn(0x7f26ec0f3450 msgr2=0x7f26ec0f5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:38:42.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 shutdown_connections 2024-02-18T01:38:42.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.254+0000 7f26f0dbd700 1 -- 172.21.15.105:0/1876090042 wait complete. 2024-02-18T01:38:42.261 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_bytes = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:38:42.272 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool set-quota 6cfab109-eaaa-440b-98e0-f8450a870fb2 max_objects 0 2024-02-18T01:38:42.397 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.390+0000 7f4d55575700 1 -- 172.21.15.105:0/2504933094 >> v1:172.21.15.105:6789/0 conn(0x7f4d501042a0 legacy=0x7f4d50106740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:42.397 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.390+0000 7f4d55575700 1 -- 172.21.15.105:0/2504933094 shutdown_connections 2024-02-18T01:38:42.397 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.390+0000 7f4d55575700 1 -- 172.21.15.105:0/2504933094 >> 172.21.15.105:0/2504933094 conn(0x7f4d500f35d0 msgr2=0x7f4d500f59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:38:42.397 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- 172.21.15.105:0/2504933094 shutdown_connections 2024-02-18T01:38:42.397 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- 172.21.15.105:0/2504933094 wait complete. 2024-02-18T01:38:42.400 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 Processor -- start 2024-02-18T01:38:42.400 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- start start 2024-02-18T01:38:42.400 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4d50117180 con 0x7f4d50100790 2024-02-18T01:38:42.400 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4d50118320 con 0x7f4d501042a0 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7f4d50119520 con 0x7f4d500fd970 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d4e7fc700 1 --1- >> v1:172.21.15.105:6789/0 conn(0x7f4d50100790 0x7f4d500f9e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6789/0 says I am v1:172.21.15.105:43930/0 (socket says 172.21.15.105:43930) 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d4e7fc700 1 -- 172.21.15.105:0/648659232 learned_addr learned my addr 172.21.15.105:0/648659232 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819520626 0 0) 0x7f4d50117180 con 0x7f4d50100790 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d38003030 con 0x7f4d50100790 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1052369809 0 0) 0x7f4d50119520 con 0x7f4d500fd970 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50117180 con 0x7f4d500fd970 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3775546429 0 0) 0x7f4d50118320 con 0x7f4d501042a0 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d50119520 con 0x7f4d501042a0 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 9046985 0 0) 0x7f4d38003030 con 0x7f4d50100790 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d50118320 con 0x7f4d50100790 2024-02-18T01:38:42.401 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4d440018a0 con 0x7f4d50100790 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 857770819 0 0) 0x7f4d50117180 con 0x7f4d500fd970 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7f4d38003030 con 0x7f4d500fd970 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4d40001880 con 0x7f4d500fd970 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1860865421 0 0) 0x7f4d50118320 con 0x7f4d50100790 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 >> v1:172.21.15.105:6790/0 conn(0x7f4d500fd970 legacy=0x7f4d500f9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 >> v1:172.21.15.155:6789/0 conn(0x7f4d501042a0 legacy=0x7f4d50115900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5011a720 con 0x7f4d50100790 2024-02-18T01:38:42.402 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d50119710 con 0x7f4d50100790 2024-02-18T01:38:42.403 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d50119be0 con 0x7f4d50100790 2024-02-18T01:38:42.403 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7f4d44002c80 con 0x7f4d50100790 2024-02-18T01:38:42.403 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.394+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7f4d44005a60 con 0x7f4d50100790 2024-02-18T01:38:42.403 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.398+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7f4d44001e00 con 0x7f4d50100790 2024-02-18T01:38:42.403 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.398+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d5005f0c0 con 0x7f4d50100790 2024-02-18T01:38:42.404 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.398+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 8 ==== osd_map(742..742 src has 1..742) v4 ==== 7986+0+0 (unknown 3357935796 0 0) 0x7f4d44078c50 con 0x7f4d50100790 2024-02-18T01:38:42.404 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.398+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=743}) v3 -- 0x7f4d50118320 con 0x7f4d50100790 2024-02-18T01:38:42.407 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.402+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7f4d4404c090 con 0x7f4d50100790 2024-02-18T01:38:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[19734]: audit 2024-02-18T01:38:41.244432+0000 mon.a (mon.0) 3295 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[19734]: cluster 2024-02-18T01:38:41.244620+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T01:38:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[19734]: audit 2024-02-18T01:38:41.361162+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[19734]: audit 2024-02-18T01:38:41.361787+0000 mon.a (mon.0) 3297 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[19734]: cluster 2024-02-18T01:38:42.116256+0000 mgr.y (mgr.24475) 827 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[27669]: audit 2024-02-18T01:38:41.244432+0000 mon.a (mon.0) 3295 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[27669]: cluster 2024-02-18T01:38:41.244620+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T01:38:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[27669]: audit 2024-02-18T01:38:41.361162+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[27669]: audit 2024-02-18T01:38:41.361787+0000 mon.a (mon.0) 3297 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:42 smithi105 bash[27669]: cluster 2024-02-18T01:38:42.116256+0000 mgr.y (mgr.24475) 827 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:42.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:42 smithi155 bash[20805]: audit 2024-02-18T01:38:41.244432+0000 mon.a (mon.0) 3295 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:42.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:42 smithi155 bash[20805]: cluster 2024-02-18T01:38:41.244620+0000 mon.a (mon.0) 3296 : cluster [DBG] osdmap e741: 8 total, 8 up, 8 in 2024-02-18T01:38:42.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:42 smithi155 bash[20805]: audit 2024-02-18T01:38:41.361162+0000 mon.c (mon.2) 428 : audit [INF] from='client.? v1:172.21.15.105:0/1876090042' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:42 smithi155 bash[20805]: audit 2024-02-18T01:38:41.361787+0000 mon.a (mon.0) 3297 : audit [INF] from='client.48554 ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]: dispatch 2024-02-18T01:38:42.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:42 smithi155 bash[20805]: cluster 2024-02-18T01:38:42.116256+0000 mgr.y (mgr.24475) 827 : cluster [DBG] pgmap v1316: 188 pgs: 188 active+clean; 281 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:42.584 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:42.578+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f4d50109e40 con 0x7f4d50100790 2024-02-18T01:38:43.259 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:43.254+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 10 ==== osd_map(743..743 src has 1..743) v4 ==== 620+0+0 (unknown 1710220667 0 0) 0x7f4d4404c2c0 con 0x7f4d50100790 2024-02-18T01:38:43.260 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:43.254+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=744}) v3 -- 0x7f4d50119520 con 0x7f4d50100790 2024-02-18T01:38:43.263 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:43.258+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 11 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v743) v1 ==== 221+0+0 (unknown 517605876 0 0) 0x7f4d4404a4a0 con 0x7f4d50100790 2024-02-18T01:38:43.379 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:43.374+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_command({"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f4d500f91c0 con 0x7f4d50100790 2024-02-18T01:38:43.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[19734]: audit 2024-02-18T01:38:42.254068+0000 mon.a (mon.0) 3298 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:43.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[19734]: cluster 2024-02-18T01:38:42.254195+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T01:38:43.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[19734]: audit 2024-02-18T01:38:42.584635+0000 mon.a (mon.0) 3300 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[27669]: audit 2024-02-18T01:38:42.254068+0000 mon.a (mon.0) 3298 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[27669]: cluster 2024-02-18T01:38:42.254195+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T01:38:43.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:43 smithi105 bash[27669]: audit 2024-02-18T01:38:42.584635+0000 mon.a (mon.0) 3300 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:43.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:43 smithi155 bash[20805]: audit 2024-02-18T01:38:42.254068+0000 mon.a (mon.0) 3298 : audit [INF] from='client.48554 ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_bytes", "val": "0"}]': finished 2024-02-18T01:38:43.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:43 smithi155 bash[20805]: cluster 2024-02-18T01:38:42.254195+0000 mon.a (mon.0) 3299 : cluster [DBG] osdmap e742: 8 total, 8 up, 8 in 2024-02-18T01:38:43.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:43 smithi155 bash[20805]: audit 2024-02-18T01:38:42.584635+0000 mon.a (mon.0) 3300 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:44.269 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.266+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 12 ==== osd_map(744..744 src has 1..744) v4 ==== 620+0+0 (unknown 2856736284 0 0) 0x7f4d44051c10 con 0x7f4d50100790 2024-02-18T01:38:44.269 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.266+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 --> v1:172.21.15.105:6789/0 -- mon_subscribe({osdmap=745}) v3 -- 0x7f4d50117180 con 0x7f4d50100790 2024-02-18T01:38:44.275 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.270+0000 7f4d37fff700 1 -- 172.21.15.105:0/648659232 <== mon.0 v1:172.21.15.105:6789/0 13 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 v744) v1 ==== 221+0+0 (unknown 2221667212 0 0) 0x7f4d4404bc20 con 0x7f4d50100790 2024-02-18T01:38:44.278 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.274+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 >> v1:172.21.15.105:6800/2148176012 conn(0x7f4d38062170 legacy=0x7f4d38064630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:44.278 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.274+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 >> v1:172.21.15.105:6789/0 conn(0x7f4d50100790 legacy=0x7f4d500f9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:38:44.279 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.274+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 shutdown_connections 2024-02-18T01:38:44.279 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.274+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 >> 172.21.15.105:0/648659232 conn(0x7f4d500f35d0 msgr2=0x7f4d50107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:38:44.281 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.278+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 shutdown_connections 2024-02-18T01:38:44.281 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:38:44.278+0000 7f4d55575700 1 -- 172.21.15.105:0/648659232 wait complete. 2024-02-18T01:38:44.282 INFO:tasks.workunit.client.0.smithi105.stderr:set-quota max_objects = 0 for pool 6cfab109-eaaa-440b-98e0-f8450a870fb2 2024-02-18T01:38:44.293 INFO:tasks.workunit.client.0.smithi105.stderr:+ sleep 30 2024-02-18T01:38:44.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[19734]: audit 2024-02-18T01:38:43.263163+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:44.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[19734]: cluster 2024-02-18T01:38:43.263265+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T01:38:44.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[19734]: audit 2024-02-18T01:38:43.379784+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:44.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[19734]: cluster 2024-02-18T01:38:44.117073+0000 mgr.y (mgr.24475) 828 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-18T01:38:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[27669]: audit 2024-02-18T01:38:43.263163+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[27669]: cluster 2024-02-18T01:38:43.263265+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T01:38:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[27669]: audit 2024-02-18T01:38:43.379784+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:44.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:44 smithi105 bash[27669]: cluster 2024-02-18T01:38:44.117073+0000 mgr.y (mgr.24475) 828 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-18T01:38:44.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:44 smithi155 bash[20805]: audit 2024-02-18T01:38:43.263163+0000 mon.a (mon.0) 3301 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:44.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:44 smithi155 bash[20805]: cluster 2024-02-18T01:38:43.263265+0000 mon.a (mon.0) 3302 : cluster [DBG] osdmap e743: 8 total, 8 up, 8 in 2024-02-18T01:38:44.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:44 smithi155 bash[20805]: audit 2024-02-18T01:38:43.379784+0000 mon.a (mon.0) 3303 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]: dispatch 2024-02-18T01:38:44.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:44 smithi155 bash[20805]: cluster 2024-02-18T01:38:44.117073+0000 mgr.y (mgr.24475) 828 : cluster [DBG] pgmap v1319: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.0 KiB/s wr, 0 op/s 2024-02-18T01:38:45.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:45 smithi105 bash[19734]: audit 2024-02-18T01:38:44.275024+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:45.739 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:45 smithi105 bash[19734]: cluster 2024-02-18T01:38:44.275178+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T01:38:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:45 smithi105 bash[27669]: audit 2024-02-18T01:38:44.275024+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:45.740 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:45 smithi105 bash[27669]: cluster 2024-02-18T01:38:44.275178+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T01:38:45.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:45 smithi155 bash[20805]: audit 2024-02-18T01:38:44.275024+0000 mon.a (mon.0) 3304 : audit [INF] from='client.? v1:172.21.15.105:0/648659232' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "field": "max_objects", "val": "0"}]': finished 2024-02-18T01:38:45.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:45 smithi155 bash[20805]: cluster 2024-02-18T01:38:44.275178+0000 mon.a (mon.0) 3305 : cluster [DBG] osdmap e744: 8 total, 8 up, 8 in 2024-02-18T01:38:46.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:46 smithi105 bash[27669]: cluster 2024-02-18T01:38:46.117885+0000 mgr.y (mgr.24475) 829 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-18T01:38:46.678 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:46 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:46] "GET /metrics HTTP/1.1" 200 191897 "" "Prometheus/2.33.4" 2024-02-18T01:38:46.678 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:46 smithi105 bash[19734]: cluster 2024-02-18T01:38:46.117885+0000 mgr.y (mgr.24475) 829 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-18T01:38:46.743 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:46 smithi155 bash[20805]: cluster 2024-02-18T01:38:46.117885+0000 mgr.y (mgr.24475) 829 : cluster [DBG] pgmap v1321: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.5 KiB/s wr, 1 op/s 2024-02-18T01:38:48.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:48 smithi105 bash[19734]: cluster 2024-02-18T01:38:48.119801+0000 mgr.y (mgr.24475) 830 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:38:48.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:48 smithi105 bash[27669]: cluster 2024-02-18T01:38:48.119801+0000 mgr.y (mgr.24475) 830 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:38:48.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:48 smithi155 bash[20805]: cluster 2024-02-18T01:38:48.119801+0000 mgr.y (mgr.24475) 830 : cluster [DBG] pgmap v1322: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-02-18T01:38:48.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:48 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:49.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:49 smithi105 bash[27669]: audit 2024-02-18T01:38:48.392002+0000 mgr.y (mgr.24475) 831 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:49.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:49 smithi105 bash[19734]: audit 2024-02-18T01:38:48.392002+0000 mgr.y (mgr.24475) 831 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:49.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:49 smithi155 bash[20805]: audit 2024-02-18T01:38:48.392002+0000 mgr.y (mgr.24475) 831 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:50.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:50 smithi105 bash[19734]: cluster 2024-02-18T01:38:50.120662+0000 mgr.y (mgr.24475) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-18T01:38:50.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:50 smithi105 bash[27669]: cluster 2024-02-18T01:38:50.120662+0000 mgr.y (mgr.24475) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-18T01:38:50.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:50 smithi155 bash[20805]: cluster 2024-02-18T01:38:50.120662+0000 mgr.y (mgr.24475) 832 : cluster [DBG] pgmap v1323: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-02-18T01:38:52.218 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:38:51 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:51] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:38:52.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:52 smithi105 bash[27669]: cluster 2024-02-18T01:38:52.121504+0000 mgr.y (mgr.24475) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 577 B/s rd, 0 op/s 2024-02-18T01:38:52.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:52 smithi105 bash[19734]: cluster 2024-02-18T01:38:52.121504+0000 mgr.y (mgr.24475) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 577 B/s rd, 0 op/s 2024-02-18T01:38:52.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:52 smithi155 bash[20805]: cluster 2024-02-18T01:38:52.121504+0000 mgr.y (mgr.24475) 833 : cluster [DBG] pgmap v1324: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 577 B/s rd, 0 op/s 2024-02-18T01:38:54.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:54 smithi105 bash[19734]: cluster 2024-02-18T01:38:54.123514+0000 mgr.y (mgr.24475) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:54.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:54 smithi105 bash[27669]: cluster 2024-02-18T01:38:54.123514+0000 mgr.y (mgr.24475) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:54.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:54 smithi155 bash[20805]: cluster 2024-02-18T01:38:54.123514+0000 mgr.y (mgr.24475) 834 : cluster [DBG] pgmap v1325: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:38:56.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:56 smithi105 bash[27669]: cluster 2024-02-18T01:38:56.124411+0000 mgr.y (mgr.24475) 835 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-02-18T01:38:56.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:56 smithi105 bash[19734]: cluster 2024-02-18T01:38:56.124411+0000 mgr.y (mgr.24475) 835 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-02-18T01:38:56.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:56 smithi155 bash[20805]: cluster 2024-02-18T01:38:56.124411+0000 mgr.y (mgr.24475) 835 : cluster [DBG] pgmap v1326: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 863 B/s rd, 0 op/s 2024-02-18T01:38:56.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:38:56 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:38:56] "GET /metrics HTTP/1.1" 200 191900 "" "Prometheus/2.33.4" 2024-02-18T01:38:58.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:58 smithi105 bash[27669]: cluster 2024-02-18T01:38:58.126397+0000 mgr.y (mgr.24475) 836 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:58.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:58 smithi105 bash[19734]: cluster 2024-02-18T01:38:58.126397+0000 mgr.y (mgr.24475) 836 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:58.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:58 smithi155 bash[20805]: cluster 2024-02-18T01:38:58.126397+0000 mgr.y (mgr.24475) 836 : cluster [DBG] pgmap v1327: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:38:58.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:38:58 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:38:59.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:38:59 smithi105 bash[27669]: audit 2024-02-18T01:38:58.395990+0000 mgr.y (mgr.24475) 837 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:59.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:38:59 smithi105 bash[19734]: audit 2024-02-18T01:38:58.395990+0000 mgr.y (mgr.24475) 837 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:38:59.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:38:59 smithi155 bash[20805]: audit 2024-02-18T01:38:58.395990+0000 mgr.y (mgr.24475) 837 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:39:00.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:00 smithi105 bash[19734]: cluster 2024-02-18T01:39:00.127202+0000 mgr.y (mgr.24475) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:00.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:00 smithi105 bash[27669]: cluster 2024-02-18T01:39:00.127202+0000 mgr.y (mgr.24475) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:00.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:00 smithi155 bash[20805]: cluster 2024-02-18T01:39:00.127202+0000 mgr.y (mgr.24475) 838 : cluster [DBG] pgmap v1328: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:02.218 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:39:01 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:39:01] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:39:02.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:02 smithi105 bash[27669]: cluster 2024-02-18T01:39:02.128095+0000 mgr.y (mgr.24475) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:02.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:02 smithi105 bash[19734]: cluster 2024-02-18T01:39:02.128095+0000 mgr.y (mgr.24475) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:02.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:02 smithi155 bash[20805]: cluster 2024-02-18T01:39:02.128095+0000 mgr.y (mgr.24475) 839 : cluster [DBG] pgmap v1329: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:04.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:04 smithi105 bash[19734]: cluster 2024-02-18T01:39:04.130141+0000 mgr.y (mgr.24475) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:04.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:04 smithi105 bash[27669]: cluster 2024-02-18T01:39:04.130141+0000 mgr.y (mgr.24475) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:04.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:04 smithi155 bash[20805]: cluster 2024-02-18T01:39:04.130141+0000 mgr.y (mgr.24475) 840 : cluster [DBG] pgmap v1330: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:06.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:06 smithi105 bash[27669]: cluster 2024-02-18T01:39:06.130949+0000 mgr.y (mgr.24475) 841 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:06.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:06 smithi105 bash[19734]: cluster 2024-02-18T01:39:06.130949+0000 mgr.y (mgr.24475) 841 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:06.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:06 smithi155 bash[20805]: cluster 2024-02-18T01:39:06.130949+0000 mgr.y (mgr.24475) 841 : cluster [DBG] pgmap v1331: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:06.989 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:06 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:39:06] "GET /metrics HTTP/1.1" 200 191900 "" "Prometheus/2.33.4" 2024-02-18T01:39:08.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:08 smithi105 bash[19734]: cluster 2024-02-18T01:39:08.132819+0000 mgr.y (mgr.24475) 842 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:08.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:08 smithi105 bash[27669]: cluster 2024-02-18T01:39:08.132819+0000 mgr.y (mgr.24475) 842 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:08.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:08 smithi155 bash[20805]: cluster 2024-02-18T01:39:08.132819+0000 mgr.y (mgr.24475) 842 : cluster [DBG] pgmap v1332: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:08.493 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:39:08 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:39:09.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:09 smithi105 bash[19734]: audit 2024-02-18T01:39:08.407211+0000 mgr.y (mgr.24475) 843 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:39:09.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:09 smithi105 bash[27669]: audit 2024-02-18T01:39:08.407211+0000 mgr.y (mgr.24475) 843 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:39:09.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:09 smithi155 bash[20805]: audit 2024-02-18T01:39:08.407211+0000 mgr.y (mgr.24475) 843 : audit [DBG] from='client.24448 v1:172.21.15.155:0/415076281' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-02-18T01:39:10.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:10 smithi105 bash[27669]: cluster 2024-02-18T01:39:10.133666+0000 mgr.y (mgr.24475) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:10.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:10 smithi105 bash[19734]: cluster 2024-02-18T01:39:10.133666+0000 mgr.y (mgr.24475) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:10.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:10 smithi155 bash[20805]: cluster 2024-02-18T01:39:10.133666+0000 mgr.y (mgr.24475) 844 : cluster [DBG] pgmap v1333: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:12.221 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:39:11 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:39:11] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:39:12.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:12 smithi105 bash[27669]: cluster 2024-02-18T01:39:12.134507+0000 mgr.y (mgr.24475) 845 : cluster [DBG] pgmap v1334: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:12.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:12 smithi105 bash[19734]: cluster 2024-02-18T01:39:12.134507+0000 mgr.y (mgr.24475) 845 : cluster [DBG] pgmap v1334: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:12.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:12 smithi155 bash[20805]: cluster 2024-02-18T01:39:12.134507+0000 mgr.y (mgr.24475) 845 : cluster [DBG] pgmap v1334: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-02-18T01:39:14.294 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool delete 6cfab109-eaaa-440b-98e0-f8450a870fb2 6cfab109-eaaa-440b-98e0-f8450a870fb2 --yes-i-really-really-mean-it 2024-02-18T01:39:14.426 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- 172.21.15.105:0/3113803437 >> v1:172.21.15.105:6790/0 conn(0x7fa028103fc0 legacy=0x7fa0281043a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:14.426 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- 172.21.15.105:0/3113803437 shutdown_connections 2024-02-18T01:39:14.426 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- 172.21.15.105:0/3113803437 >> 172.21.15.105:0/3113803437 conn(0x7fa02806d830 msgr2=0x7fa02806fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:39:14.427 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- 172.21.15.105:0/3113803437 shutdown_connections 2024-02-18T01:39:14.427 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- 172.21.15.105:0/3113803437 wait complete. 2024-02-18T01:39:14.427 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 Processor -- start 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- start start 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa02811d6e0 con 0x7fa028103fc0 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa02811e8c0 con 0x7fa028106d60 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa030631700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fa02811fac0 con 0x7fa02810a8a0 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa02ebce700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fa02810a8a0 0x7fa02811bf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:58508/0 (socket says 172.21.15.105:58508) 2024-02-18T01:39:14.428 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa02ebce700 1 -- 172.21.15.105:0/1249407397 learned_addr learned my addr 172.21.15.105:0/1249407397 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:39:14.430 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2895651947 0 0) 0x7fa02811d6e0 con 0x7fa028103fc0 2024-02-18T01:39:14.430 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa014003030 con 0x7fa028103fc0 2024-02-18T01:39:14.430 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1490868584 0 0) 0x7fa02811fac0 con 0x7fa02810a8a0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa02811d6e0 con 0x7fa02810a8a0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052650054 0 0) 0x7fa02811e8c0 con 0x7fa028106d60 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa02811fac0 con 0x7fa028106d60 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.1 v1:172.21.15.155:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1766818033 0 0) 0x7fa02811fac0 con 0x7fa028106d60 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.155:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa02811e8c0 con 0x7fa028106d60 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.421+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.1 v1:172.21.15.155:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa01c001880 con 0x7fa028106d60 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2074103261 0 0) 0x7fa02811d6e0 con 0x7fa02810a8a0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa02811fac0 con 0x7fa02810a8a0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa020001c20 con 0x7fa02810a8a0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 1591094322 0 0) 0x7fa014003030 con 0x7fa028103fc0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fa02811d6e0 con 0x7fa028103fc0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa024001570 con 0x7fa028103fc0 2024-02-18T01:39:14.431 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 2357705026 0 0) 0x7fa02811fac0 con 0x7fa02810a8a0 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 >> v1:172.21.15.155:6789/0 conn(0x7fa028106d60 legacy=0x7fa028100430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 >> v1:172.21.15.105:6789/0 conn(0x7fa028103fc0 legacy=0x7fa0280ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa028120cc0 con 0x7fa02810a8a0 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa02811d8d0 con 0x7fa02810a8a0 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fa02811dda0 con 0x7fa02810a8a0 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fa020002330 con 0x7fa02810a8a0 2024-02-18T01:39:14.432 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fa020005db0 con 0x7fa02810a8a0 2024-02-18T01:39:14.433 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa00c004ee0 con 0x7fa02810a8a0 2024-02-18T01:39:14.433 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fa020001e90 con 0x7fa02810a8a0 2024-02-18T01:39:14.433 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(744..744 src has 1..744) v4 ==== 7986+0+0 (unknown 2086683151 0 0) 0x7fa020078ec0 con 0x7fa02810a8a0 2024-02-18T01:39:14.433 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.425+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=745}) v3 -- 0x7fa02811fac0 con 0x7fa02810a8a0 2024-02-18T01:39:14.440 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.433+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fa02004c300 con 0x7fa02810a8a0 2024-02-18T01:39:14.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:14 smithi105 bash[19734]: cluster 2024-02-18T01:39:14.136554+0000 mgr.y (mgr.24475) 846 : cluster [DBG] pgmap v1335: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:14.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:14 smithi105 bash[27669]: cluster 2024-02-18T01:39:14.136554+0000 mgr.y (mgr.24475) 846 : cluster [DBG] pgmap v1335: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:14.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:14 smithi155 bash[20805]: cluster 2024-02-18T01:39:14.136554+0000 mgr.y (mgr.24475) 846 : cluster [DBG] pgmap v1335: 188 pgs: 188 active+clean; 292 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-02-18T01:39:14.615 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:14.609+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa00c005710 con 0x7fa02810a8a0 2024-02-18T01:39:15.231 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.225+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 10 ==== osd_map(745..745 src has 1..745) v4 ==== 288+0+0 (unknown 3890780064 0 0) 0x7fa02004c530 con 0x7fa02810a8a0 2024-02-18T01:39:15.231 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.225+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=746}) v3 -- 0x7fa02811e8c0 con 0x7fa02810a8a0 2024-02-18T01:39:15.234 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.229+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]=0 pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' removed v745) v1 ==== 248+0+0 (unknown 107203507 0 0) 0x7fa02004a710 con 0x7fa02810a8a0 2024-02-18T01:39:15.355 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.349+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa00c005a10 con 0x7fa02810a8a0 2024-02-18T01:39:15.356 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.349+0000 7fa01b7fe700 1 -- 172.21.15.105:0/1249407397 <== mon.2 v1:172.21.15.105:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]=0 pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' does not exist v745) v1 ==== 255+0+0 (unknown 3233854744 0 0) 0x7fa02004be90 con 0x7fa02810a8a0 2024-02-18T01:39:15.358 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 >> v1:172.21.15.105:6800/2148176012 conn(0x7fa014062480 legacy=0x7fa014064940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:15.358 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 >> v1:172.21.15.105:6790/0 conn(0x7fa02810a8a0 legacy=0x7fa02811bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:15.359 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 shutdown_connections 2024-02-18T01:39:15.359 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 >> 172.21.15.105:0/1249407397 conn(0x7fa02806d830 msgr2=0x7fa0281065d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:39:15.359 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 shutdown_connections 2024-02-18T01:39:15.359 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.353+0000 7fa030631700 1 -- 172.21.15.105:0/1249407397 wait complete. 2024-02-18T01:39:15.360 INFO:tasks.workunit.client.0.smithi105.stderr:pool '6cfab109-eaaa-440b-98e0-f8450a870fb2' does not exist 2024-02-18T01:39:15.370 INFO:tasks.workunit.client.0.smithi105.stderr:+ ceph osd pool delete 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5 --yes-i-really-really-mean-it 2024-02-18T01:39:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:15 smithi105 bash[19734]: audit 2024-02-18T01:39:14.616189+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.489 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:15 smithi105 bash[19734]: audit 2024-02-18T01:39:14.616673+0000 mon.a (mon.0) 3306 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:15 smithi105 bash[27669]: audit 2024-02-18T01:39:14.616189+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:15 smithi105 bash[27669]: audit 2024-02-18T01:39:14.616673+0000 mon.a (mon.0) 3306 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:15 smithi155 bash[20805]: audit 2024-02-18T01:39:14.616189+0000 mon.c (mon.2) 429 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:15 smithi155 bash[20805]: audit 2024-02-18T01:39:14.616673+0000 mon.a (mon.0) 3306 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:15.495 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3879492477 >> v1:172.21.15.105:6790/0 conn(0x7fbd580fee10 legacy=0x7fbd58101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:15.495 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3879492477 shutdown_connections 2024-02-18T01:39:15.495 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3879492477 >> 172.21.15.105:0/3879492477 conn(0x7fbd5806c100 msgr2=0x7fbd5806c510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:39:15.496 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3879492477 shutdown_connections 2024-02-18T01:39:15.496 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3879492477 wait complete. 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 Processor -- start 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- start start 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- --> v1:172.21.15.105:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd580fb840 con 0x7fbd580fbfc0 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- --> v1:172.21.15.155:6789/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd580fba30 con 0x7fbd580fee10 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5f09b700 1 -- --> v1:172.21.15.105:6790/0 -- auth(proto 0 34 bytes epoch 0) v1 -- 0x7fbd5811baf0 con 0x7fbd58102920 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5d638700 1 --1- >> v1:172.21.15.105:6790/0 conn(0x7fbd58102920 0x7fbd581183b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=1).handle_server_banner_and_identify peer v1:172.21.15.105:6790/0 says I am v1:172.21.15.105:58528/0 (socket says 172.21.15.105:58528) 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd5d638700 1 -- 172.21.15.105:0/3800995736 learned_addr learned my addr 172.21.15.105:0/3800995736 (peer_addr_for_me v1:172.21.15.105:0/0) 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621754682 0 0) 0x7fbd5811baf0 con 0x7fbd58102920 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd480035f0 con 0x7fbd58102920 2024-02-18T01:39:15.498 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.0 v1:172.21.15.105:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2911529957 0 0) 0x7fbd580fb840 con 0x7fbd580fbfc0 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.489+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd5811baf0 con 0x7fbd580fbfc0 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.1 v1:172.21.15.155:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2285383265 0 0) 0x7fbd580fba30 con 0x7fbd580fee10 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.155:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd580fb840 con 0x7fbd580fee10 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 2439291330 0 0) 0x7fbd480035f0 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd580fba30 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbd50002c50 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.0 v1:172.21.15.105:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 764+0+0 (unknown 423675180 0 0) 0x7fbd5811baf0 con 0x7fbd580fbfc0 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6789/0 -- auth(proto 2 165 bytes epoch 0) v1 -- 0x7fbd480035f0 con 0x7fbd580fbfc0 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.0 v1:172.21.15.105:6789/0 3 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbd4c001840 con 0x7fbd580fbfc0 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 580+0+0 (unknown 1926465727 0 0) 0x7fbd580fba30 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 >> v1:172.21.15.155:6789/0 conn(0x7fbd580fee10 legacy=0x7fbd580f8340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 >> v1:172.21.15.105:6789/0 conn(0x7fbd580fbfc0 legacy=0x7fbd580f7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5811cc90 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd5811bce0 con 0x7fbd58102920 2024-02-18T01:39:15.499 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5811c130 con 0x7fbd58102920 2024-02-18T01:39:15.500 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 5 ==== config(24 keys) v1 ==== 1067+0+0 (unknown 3036552504 0 0) 0x7fbd50003000 con 0x7fbd58102920 2024-02-18T01:39:15.500 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 6 ==== mon_map magic: 0 v1 ==== 308+0+0 (unknown 3175259303 0 0) 0x7fbd50005db0 con 0x7fbd58102920 2024-02-18T01:39:15.500 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd3c004ee0 con 0x7fbd58102920 2024-02-18T01:39:15.500 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 7 ==== mgrmap(e 19) v1 ==== 77088+0+0 (unknown 1882041087 0 0) 0x7fbd50018cc0 con 0x7fbd58102920 2024-02-18T01:39:15.501 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 8 ==== osd_map(745..745 src has 1..745) v4 ==== 7598+0+0 (unknown 1239382228 0 0) 0x7fbd50078d90 con 0x7fbd58102920 2024-02-18T01:39:15.501 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.493+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=746}) v3 -- 0x7fbd580fba30 con 0x7fbd58102920 2024-02-18T01:39:15.507 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.501+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 9 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+163658 (unknown 1092875540 0 292100661) 0x7fbd5004c3b0 con 0x7fbd58102920 2024-02-18T01:39:15.682 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:15.677+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd3c005710 con 0x7fbd58102920 2024-02-18T01:39:16.243 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.237+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 10 ==== osd_map(746..746 src has 1..746) v4 ==== 288+0+0 (unknown 3193064615 0 0) 0x7fbd500a3f60 con 0x7fbd58102920 2024-02-18T01:39:16.243 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.237+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_subscribe({osdmap=747}) v3 -- 0x7fbd480035f0 con 0x7fbd58102920 2024-02-18T01:39:16.248 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.241+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 11 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]=0 pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' removed v746) v1 ==== 248+0+0 (unknown 1105313008 0 0) 0x7fbd5004a7c0 con 0x7fbd58102920 2024-02-18T01:39:16.367 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.361+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 --> v1:172.21.15.105:6790/0 -- mon_command({"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd3c005a10 con 0x7fbd58102920 2024-02-18T01:39:16.368 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.361+0000 7fbd55ffb700 1 -- 172.21.15.105:0/3800995736 <== mon.2 v1:172.21.15.105:6790/0 12 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]=0 pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' does not exist v746) v1 ==== 255+0+0 (unknown 555750639 0 0) 0x7fbd5004bf40 con 0x7fbd58102920 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 >> v1:172.21.15.105:6800/2148176012 conn(0x7fbd480621b0 legacy=0x7fbd48064670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 >> v1:172.21.15.105:6790/0 conn(0x7fbd58102920 legacy=0x7fbd581183b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 shutdown_connections 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 >> 172.21.15.105:0/3800995736 conn(0x7fbd5806c100 msgr2=0x7fbd5806c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 shutdown_connections 2024-02-18T01:39:16.371 INFO:tasks.workunit.client.0.smithi105.stderr:2024-02-18T01:39:16.365+0000 7fbd5f09b700 1 -- 172.21.15.105:0/3800995736 wait complete. 2024-02-18T01:39:16.372 INFO:tasks.workunit.client.0.smithi105.stderr:pool '7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5' does not exist 2024-02-18T01:39:16.382 INFO:tasks.workunit.client.0.smithi105.stderr:+ echo OK 2024-02-18T01:39:16.383 INFO:tasks.workunit.client.0.smithi105.stdout:OK 2024-02-18T01:39:16.384 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-02-18T01:39:16.384 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-02-18T01:39:16.440 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-02-18T01:39:16.440 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-02-18T01:39:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: audit 2024-02-18T01:39:15.233843+0000 mon.a (mon.0) 3307 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:39:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: cluster 2024-02-18T01:39:15.233943+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T01:39:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: audit 2024-02-18T01:39:15.355526+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.489 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: audit 2024-02-18T01:39:15.356020+0000 mon.a (mon.0) 3309 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: audit 2024-02-18T01:39:15.682905+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.105:0/3800995736' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: audit 2024-02-18T01:39:15.683561+0000 mon.a (mon.0) 3310 : audit [INF] from='client.48572 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[27669]: cluster 2024-02-18T01:39:16.137375+0000 mgr.y (mgr.24475) 847 : cluster [DBG] pgmap v1337: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: audit 2024-02-18T01:39:15.233843+0000 mon.a (mon.0) 3307 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: cluster 2024-02-18T01:39:15.233943+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T01:39:16.490 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: audit 2024-02-18T01:39:15.355526+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: audit 2024-02-18T01:39:15.356020+0000 mon.a (mon.0) 3309 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: audit 2024-02-18T01:39:15.682905+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.105:0/3800995736' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: audit 2024-02-18T01:39:15.683561+0000 mon.a (mon.0) 3310 : audit [INF] from='client.48572 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.491 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[19734]: cluster 2024-02-18T01:39:16.137375+0000 mgr.y (mgr.24475) 847 : cluster [DBG] pgmap v1337: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:39:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: audit 2024-02-18T01:39:15.233843+0000 mon.a (mon.0) 3307 : audit [INF] from='client.48566 ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]': finished 2024-02-18T01:39:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: cluster 2024-02-18T01:39:15.233943+0000 mon.a (mon.0) 3308 : cluster [DBG] osdmap e745: 8 total, 8 up, 8 in 2024-02-18T01:39:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: audit 2024-02-18T01:39:15.355526+0000 mon.c (mon.2) 430 : audit [INF] from='client.? v1:172.21.15.105:0/1249407397' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.493 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: audit 2024-02-18T01:39:15.356020+0000 mon.a (mon.0) 3309 : audit [INF] from='client.48566 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "pool2": "6cfab109-eaaa-440b-98e0-f8450a870fb2", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: audit 2024-02-18T01:39:15.682905+0000 mon.c (mon.2) 431 : audit [INF] from='client.? v1:172.21.15.105:0/3800995736' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: audit 2024-02-18T01:39:15.683561+0000 mon.a (mon.0) 3310 : audit [INF] from='client.48572 ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "pool2": "7140d1a6-1fae-4b60-aa76-86d2d8fdb1b5", "yes_i_really_really_mean_it": true}]: dispatch 2024-02-18T01:39:16.494 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:16 smithi155 bash[20805]: cluster 2024-02-18T01:39:16.137375+0000 mgr.y (mgr.24475) 847 : cluster [DBG] pgmap v1337: 176 pgs: 176 active+clean; 120 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-02-18T01:39:16.726 DEBUG:teuthology.parallel:result is None 2024-02-18T01:39:16.727 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T01:39:16.777 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:16 smithi105 bash[20046]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:39:16] "GET /metrics HTTP/1.1" 200 191900 "" "Prometheus/2.33.4" 2024-02-18T01:39:16.781 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T01:39:16.781 DEBUG:teuthology.orchestra.run.smithi105:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-02-18T01:39:16.852 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-02-18T01:39:16.852 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-02-18T01:39:16.864 INFO:tasks.cephadm:Teardown begin 2024-02-18T01:39:16.865 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:39:16.908 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:39:16.922 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-02-18T01:39:16.922 DEBUG:teuthology.orchestra.run.smithi105:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T01:39:16.954 DEBUG:teuthology.orchestra.run.smithi155:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-02-18T01:39:16.970 INFO:tasks.cephadm:Stopping all daemons... 2024-02-18T01:39:16.970 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-02-18T01:39:16.971 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a 2024-02-18T01:39:17.041 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:17 smithi105 systemd[1]: Stopping Ceph mon.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:17.335 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[129013]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mon.a 2024-02-18T01:39:17.335 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[19734]: debug 2024-02-18T01:39:17.201+0000 7fbf66470700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T01:39:17.335 INFO:journalctl@ceph.mon.a.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[19734]: debug 2024-02-18T01:39:17.201+0000 7fbf66470700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-02-18T01:39:17.335 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE Bus STOPPING 2024-02-18T01:39:17.335 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-02-18T01:39:17.336 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE Bus STOPPED 2024-02-18T01:39:17.336 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE Bus STARTING 2024-02-18T01:39:17.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE Serving on http://:::9283 2024-02-18T01:39:17.739 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:17 smithi105 bash[20046]: [18/Feb/2024:01:39:17] ENGINE Bus STARTED 2024-02-18T01:39:18.369 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.a.service' 2024-02-18T01:39:18.395 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:18.396 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-02-18T01:39:18.396 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-02-18T01:39:18.396 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.c 2024-02-18T01:39:18.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:18 smithi105 systemd[1]: Stopping Ceph mon.c for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:18.678 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:18 smithi105 bash[129153]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mon.c 2024-02-18T01:39:18.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:39:18 smithi155 bash[43865]: debug there is no tcmu-runner data available 2024-02-18T01:39:18.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:18 smithi105 bash[27669]: debug 2024-02-18T01:39:18.673+0000 7f9fdd9e4700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T01:39:18.989 INFO:journalctl@ceph.mon.c.smithi105.stdout:Feb 18 01:39:18 smithi105 bash[27669]: debug 2024-02-18T01:39:18.673+0000 7f9fdd9e4700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-02-18T01:39:19.791 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.c.service' 2024-02-18T01:39:19.815 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:19.815 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-02-18T01:39:19.815 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-02-18T01:39:19.816 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.b 2024-02-18T01:39:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:19 smithi155 systemd[1]: Stopping Ceph mon.b for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:19 smithi155 bash[49859]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mon.b 2024-02-18T01:39:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:20 smithi155 bash[20805]: debug 2024-02-18T01:39:20.024+0000 7f68a8347700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-02-18T01:39:20.243 INFO:journalctl@ceph.mon.b.smithi155.stdout:Feb 18 01:39:20 smithi155 bash[20805]: debug 2024-02-18T01:39:20.024+0000 7f68a8347700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-02-18T01:39:21.085 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mon.b.service' 2024-02-18T01:39:21.111 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:21.111 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-02-18T01:39:21.111 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-02-18T01:39:21.112 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y 2024-02-18T01:39:21.376 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:21 smithi105 systemd[1]: Stopping Ceph mgr.y for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:21.376 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:21 smithi105 bash[129280]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mgr.y 2024-02-18T01:39:21.376 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:21 smithi105 bash[20046]: debug 2024-02-18T01:39:21.309+0000 7f74a8ea6700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:21.376 INFO:journalctl@ceph.mgr.y.smithi105.stdout:Feb 18 01:39:21 smithi105 bash[20046]: debug 2024-02-18T01:39:21.309+0000 7f74a8ea6700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-02-18T01:39:22.171 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.y.service' 2024-02-18T01:39:22.191 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:22.191 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-02-18T01:39:22.191 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-02-18T01:39:22.191 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.x 2024-02-18T01:39:22.204 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:39:21 smithi155 bash[22101]: ::ffff:172.21.15.155 - - [18/Feb/2024:01:39:21] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.33.4" 2024-02-18T01:39:22.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:39:22 smithi155 systemd[1]: Stopping Ceph mgr.x for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:22.484 INFO:journalctl@ceph.mgr.x.smithi155.stdout:Feb 18 01:39:22 smithi155 bash[49992]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-mgr.x 2024-02-18T01:39:23.287 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@mgr.x.service' 2024-02-18T01:39:23.358 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:23.358 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-02-18T01:39:23.359 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-02-18T01:39:23.359 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.0 2024-02-18T01:39:23.739 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:23.576Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:23.739 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:23.577Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:23.740 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:23 smithi105 systemd[1]: Stopping Ceph osd.0 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:23.740 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[129419]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.0 2024-02-18T01:39:23.740 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[31001]: debug 2024-02-18T01:39:23.565+0000 7f4ea61b5700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:23.740 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[31001]: debug 2024-02-18T01:39:23.565+0000 7f4ea61b5700 -1 osd.0 746 *** Got signal Terminated *** 2024-02-18T01:39:23.740 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:23 smithi105 bash[31001]: debug 2024-02-18T01:39:23.565+0000 7f4ea61b5700 -1 osd.0 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:39:29.739 INFO:journalctl@ceph.osd.0.smithi105.stdout:Feb 18 01:39:29 smithi105 bash[129419]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-0 2024-02-18T01:39:30.401 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:30 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:30.146Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:30.401 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:30 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:30.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:31.424 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.0.service' 2024-02-18T01:39:31.449 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:31.450 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-02-18T01:39:31.450 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-02-18T01:39:31.450 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.1 2024-02-18T01:39:31.683 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:31 smithi105 systemd[1]: Stopping Ceph osd.1 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:31.683 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:31 smithi105 bash[129683]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.1 2024-02-18T01:39:31.989 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:31 smithi105 bash[36150]: debug 2024-02-18T01:39:31.677+0000 7fccc0912700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:31.989 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:31 smithi105 bash[36150]: debug 2024-02-18T01:39:31.677+0000 7fccc0912700 -1 osd.1 746 *** Got signal Terminated *** 2024-02-18T01:39:31.989 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:31 smithi105 bash[36150]: debug 2024-02-18T01:39:31.677+0000 7fccc0912700 -1 osd.1 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:39:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:33 smithi105 bash[54021]: level=error ts=2024-02-18T01:39:33.576Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:33 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:33.577Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:33 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:33.578Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:37.989 INFO:journalctl@ceph.osd.1.smithi105.stdout:Feb 18 01:39:37 smithi105 bash[129683]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-1 2024-02-18T01:39:39.581 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.1.service' 2024-02-18T01:39:39.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:39.605 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-02-18T01:39:39.605 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-02-18T01:39:39.606 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.2 2024-02-18T01:39:39.853 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:39 smithi105 systemd[1]: Stopping Ceph osd.2 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:39.853 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:39 smithi105 bash[129935]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.2 2024-02-18T01:39:40.147 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:39 smithi105 bash[41139]: debug 2024-02-18T01:39:39.845+0000 7f900890f700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:40.147 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:39 smithi105 bash[41139]: debug 2024-02-18T01:39:39.845+0000 7f900890f700 -1 osd.2 746 *** Got signal Terminated *** 2024-02-18T01:39:40.147 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:39 smithi105 bash[41139]: debug 2024-02-18T01:39:39.845+0000 7f900890f700 -1 osd.2 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:39:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:40 smithi105 bash[54021]: level=error ts=2024-02-18T01:39:40.146Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:40 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:40.147Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:40 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:40.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:43.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:43 smithi105 bash[54021]: level=error ts=2024-02-18T01:39:43.576Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:43.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:43 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:43.578Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:43.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:43 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:43.579Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:45.989 INFO:journalctl@ceph.osd.2.smithi105.stdout:Feb 18 01:39:45 smithi105 bash[129935]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-2 2024-02-18T01:39:47.700 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.2.service' 2024-02-18T01:39:47.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:47.724 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-02-18T01:39:47.724 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-02-18T01:39:47.724 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.3 2024-02-18T01:39:47.970 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:47 smithi105 systemd[1]: Stopping Ceph osd.3 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:47.970 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:47 smithi105 bash[130197]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.3 2024-02-18T01:39:48.239 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:47 smithi105 bash[46332]: debug 2024-02-18T01:39:47.965+0000 7f4b8300b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:48.239 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:47 smithi105 bash[46332]: debug 2024-02-18T01:39:47.965+0000 7f4b8300b700 -1 osd.3 746 *** Got signal Terminated *** 2024-02-18T01:39:48.239 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:47 smithi105 bash[46332]: debug 2024-02-18T01:39:47.965+0000 7f4b8300b700 -1 osd.3 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:39:49.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:49 smithi155 bash[40046]: debug 2024-02-18T01:39:49.384+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:50.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:49 smithi155 bash[24809]: debug 2024-02-18T01:39:49.748+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:50.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:49 smithi155 bash[29937]: debug 2024-02-18T01:39:49.920+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:50.489 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:50 smithi105 bash[46332]: debug 2024-02-18T01:39:50.089+0000 7f4b7f787700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:27.203494+0000 front 2024-02-18T01:39:27.203683+0000 (oldest deadline 2024-02-18T01:39:49.503706+0000) 2024-02-18T01:39:50.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:50 smithi105 bash[54021]: level=error ts=2024-02-18T01:39:50.147Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:50.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:50 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:50.148Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:50.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:50 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:50.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:50.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:50 smithi155 bash[40046]: debug 2024-02-18T01:39:50.408+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:50.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:50 smithi155 bash[34908]: debug 2024-02-18T01:39:50.556+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:51.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:50 smithi155 bash[24809]: debug 2024-02-18T01:39:50.764+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:51.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:50 smithi155 bash[29937]: debug 2024-02-18T01:39:50.940+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:51.489 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:51 smithi105 bash[46332]: debug 2024-02-18T01:39:51.105+0000 7f4b7f787700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:27.203494+0000 front 2024-02-18T01:39:27.203683+0000 (oldest deadline 2024-02-18T01:39:49.503706+0000) 2024-02-18T01:39:51.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:51 smithi155 bash[40046]: debug 2024-02-18T01:39:51.432+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:51.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:51 smithi155 bash[34908]: debug 2024-02-18T01:39:51.548+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:52.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:51 smithi155 bash[24809]: debug 2024-02-18T01:39:51.788+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:52.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:51 smithi155 bash[29937]: debug 2024-02-18T01:39:51.940+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:52.489 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:52 smithi105 bash[46332]: debug 2024-02-18T01:39:52.089+0000 7f4b7f787700 -1 osd.3 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:27.203494+0000 front 2024-02-18T01:39:27.203683+0000 (oldest deadline 2024-02-18T01:39:49.503706+0000) 2024-02-18T01:39:52.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:52 smithi155 bash[34908]: debug 2024-02-18T01:39:52.560+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:52.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:52 smithi155 bash[40046]: debug 2024-02-18T01:39:52.404+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:53.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:52 smithi155 bash[29937]: debug 2024-02-18T01:39:52.968+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:53.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:52 smithi155 bash[24809]: debug 2024-02-18T01:39:52.756+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:53.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:53 smithi155 bash[34908]: debug 2024-02-18T01:39:53.604+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:53.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:53 smithi155 bash[40046]: debug 2024-02-18T01:39:53.384+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:53.989 INFO:journalctl@ceph.osd.3.smithi105.stdout:Feb 18 01:39:53 smithi105 bash[130197]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-3 2024-02-18T01:39:53.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:53 smithi105 bash[54021]: level=error ts=2024-02-18T01:39:53.577Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:53.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:53 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:53.577Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:39:53.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:39:53 smithi105 bash[54021]: level=warn ts=2024-02-18T01:39:53.579Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:39:54.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:53 smithi155 bash[24809]: debug 2024-02-18T01:39:53.784+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:54.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:53 smithi155 bash[29937]: debug 2024-02-18T01:39:53.964+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:54.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:54 smithi155 bash[34908]: debug 2024-02-18T01:39:54.616+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:54.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:54 smithi155 bash[40046]: debug 2024-02-18T01:39:54.428+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:55.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:54 smithi155 bash[24809]: debug 2024-02-18T01:39:54.768+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:55.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[29937]: debug 2024-02-18T01:39:55.012+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:55.730 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.3.service' 2024-02-18T01:39:55.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[34908]: debug 2024-02-18T01:39:55.599+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:55.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[40046]: debug 2024-02-18T01:39:55.435+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:55.754 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:39:55.754 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-02-18T01:39:55.755 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-02-18T01:39:55.755 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.4 2024-02-18T01:39:56.244 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[29937]: debug 2024-02-18T01:39:55.963+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:56.245 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 systemd[1]: Stopping Ceph osd.4 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:39:56.245 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[50122]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.4 2024-02-18T01:39:56.245 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[24809]: debug 2024-02-18T01:39:55.799+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:56.245 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[24809]: debug 2024-02-18T01:39:55.951+0000 7fc8e0fca700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:39:56.245 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[24809]: debug 2024-02-18T01:39:55.951+0000 7fc8e0fca700 -1 osd.4 746 *** Got signal Terminated *** 2024-02-18T01:39:56.246 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:55 smithi155 bash[24809]: debug 2024-02-18T01:39:55.951+0000 7fc8e0fca700 -1 osd.4 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:39:56.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:56 smithi155 bash[34908]: debug 2024-02-18T01:39:56.603+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:56.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:56 smithi155 bash[40046]: debug 2024-02-18T01:39:56.451+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:57.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:56 smithi155 bash[24809]: debug 2024-02-18T01:39:56.795+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:57.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:56 smithi155 bash[29937]: debug 2024-02-18T01:39:56.975+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:57.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:57 smithi155 bash[40046]: debug 2024-02-18T01:39:57.487+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:57.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:57 smithi155 bash[34908]: debug 2024-02-18T01:39:57.571+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:57.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:57 smithi155 bash[34908]: debug 2024-02-18T01:39:57.571+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:39:58.021 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:57 smithi155 bash[24809]: debug 2024-02-18T01:39:57.763+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:58.493 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[29937]: debug 2024-02-18T01:39:58.015+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:58.493 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[29937]: debug 2024-02-18T01:39:58.015+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:39:58.789 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[40046]: debug 2024-02-18T01:39:58.523+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:58.790 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[34908]: debug 2024-02-18T01:39:58.623+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:58.790 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[34908]: debug 2024-02-18T01:39:58.623+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:39:59.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:58 smithi155 bash[24809]: debug 2024-02-18T01:39:58.787+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:39:59.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[29937]: debug 2024-02-18T01:39:58.999+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:39:59.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[29937]: debug 2024-02-18T01:39:58.999+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:39:59.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[34908]: debug 2024-02-18T01:39:59.651+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:39:59.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[34908]: debug 2024-02-18T01:39:59.651+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:39:59.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[40046]: debug 2024-02-18T01:39:59.483+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:39:59.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[40046]: debug 2024-02-18T01:39:59.483+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:00.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:39:59 smithi155 bash[24809]: debug 2024-02-18T01:39:59.807+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:40:00.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[29937]: debug 2024-02-18T01:40:00.019+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:00.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[29937]: debug 2024-02-18T01:40:00.019+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:00.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:00 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:00.148Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:00.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:00 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:00.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:00.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:00 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:00.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:00.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[34908]: debug 2024-02-18T01:40:00.667+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:00.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[34908]: debug 2024-02-18T01:40:00.667+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:00.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[40046]: debug 2024-02-18T01:40:00.447+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:00.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[40046]: debug 2024-02-18T01:40:00.451+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:01.091 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[24809]: debug 2024-02-18T01:40:00.823+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:24.059026+0000 front 2024-02-18T01:39:24.058629+0000 (oldest deadline 2024-02-18T01:39:49.358516+0000) 2024-02-18T01:40:01.091 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[24809]: debug 2024-02-18T01:40:00.823+0000 7fc8ddf47700 -1 osd.4 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.660009+0000 front 2024-02-18T01:39:34.660182+0000 (oldest deadline 2024-02-18T01:40:00.559918+0000) 2024-02-18T01:40:01.091 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[29937]: debug 2024-02-18T01:40:00.991+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:01.091 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:00 smithi155 bash[29937]: debug 2024-02-18T01:40:00.991+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:01.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[40046]: debug 2024-02-18T01:40:01.475+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:01.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[40046]: debug 2024-02-18T01:40:01.475+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:01.744 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[34908]: debug 2024-02-18T01:40:01.663+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:01.744 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[34908]: debug 2024-02-18T01:40:01.663+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:02.243 INFO:journalctl@ceph.osd.4.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[50122]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-4 2024-02-18T01:40:02.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[29937]: debug 2024-02-18T01:40:01.955+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:02.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:01 smithi155 bash[29937]: debug 2024-02-18T01:40:01.955+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:02.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[34908]: debug 2024-02-18T01:40:02.659+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:02.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[34908]: debug 2024-02-18T01:40:02.659+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:02.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[40046]: debug 2024-02-18T01:40:02.467+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:02.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[40046]: debug 2024-02-18T01:40:02.467+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:03.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[29937]: debug 2024-02-18T01:40:02.931+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:03.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:02 smithi155 bash[29937]: debug 2024-02-18T01:40:02.935+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:03.688 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[40046]: debug 2024-02-18T01:40:03.455+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:03.689 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[40046]: debug 2024-02-18T01:40:03.455+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:03.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:03 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:03.578Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:03.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:03 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:03.579Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:03.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:03 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:03.580Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:03.993 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[29937]: debug 2024-02-18T01:40:03.923+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:03.993 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[29937]: debug 2024-02-18T01:40:03.923+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:03.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[34908]: debug 2024-02-18T01:40:03.687+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:03.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:03 smithi155 bash[34908]: debug 2024-02-18T01:40:03.687+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:04.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.4.service' 2024-02-18T01:40:04.140 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:04.140 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-02-18T01:40:04.141 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-02-18T01:40:04.141 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.5 2024-02-18T01:40:04.367 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 systemd[1]: Stopping Ceph osd.5 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:04.367 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[50380]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.5 2024-02-18T01:40:04.692 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[40046]: debug 2024-02-18T01:40:04.443+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:04.692 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[40046]: debug 2024-02-18T01:40:04.443+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:04.692 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[29937]: debug 2024-02-18T01:40:04.363+0000 7f28a7e90700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:40:04.693 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[29937]: debug 2024-02-18T01:40:04.363+0000 7f28a7e90700 -1 osd.5 746 *** Got signal Terminated *** 2024-02-18T01:40:04.693 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[29937]: debug 2024-02-18T01:40:04.363+0000 7f28a7e90700 -1 osd.5 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:40:04.967 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[34908]: debug 2024-02-18T01:40:04.691+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:04.967 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[34908]: debug 2024-02-18T01:40:04.691+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:05.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[29937]: debug 2024-02-18T01:40:04.963+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:05.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:04 smithi155 bash[29937]: debug 2024-02-18T01:40:04.963+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:05.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[40046]: debug 2024-02-18T01:40:05.451+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:05.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[40046]: debug 2024-02-18T01:40:05.451+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:05.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[34908]: debug 2024-02-18T01:40:05.651+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:05.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[34908]: debug 2024-02-18T01:40:05.651+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:05.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[34908]: debug 2024-02-18T01:40:05.651+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:06.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[29937]: debug 2024-02-18T01:40:05.983+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:06.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:05 smithi155 bash[29937]: debug 2024-02-18T01:40:05.983+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:06.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[34908]: debug 2024-02-18T01:40:06.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:06.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[34908]: debug 2024-02-18T01:40:06.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:06.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[34908]: debug 2024-02-18T01:40:06.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:06.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[40046]: debug 2024-02-18T01:40:06.447+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:06.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[40046]: debug 2024-02-18T01:40:06.447+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:07.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[29937]: debug 2024-02-18T01:40:06.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:07.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[29937]: debug 2024-02-18T01:40:06.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:07.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:06 smithi155 bash[29937]: debug 2024-02-18T01:40:06.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:43.301574+0000 front 2024-02-18T01:39:43.301494+0000 (oldest deadline 2024-02-18T01:40:06.801459+0000) 2024-02-18T01:40:07.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[40046]: debug 2024-02-18T01:40:07.427+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:07.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[40046]: debug 2024-02-18T01:40:07.427+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:07.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[34908]: debug 2024-02-18T01:40:07.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:07.744 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[34908]: debug 2024-02-18T01:40:07.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:07.744 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[34908]: debug 2024-02-18T01:40:07.635+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:08.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[29937]: debug 2024-02-18T01:40:07.943+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:08.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[29937]: debug 2024-02-18T01:40:07.943+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:08.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:07 smithi155 bash[29937]: debug 2024-02-18T01:40:07.943+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:43.301574+0000 front 2024-02-18T01:39:43.301494+0000 (oldest deadline 2024-02-18T01:40:06.801459+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[40046]: debug 2024-02-18T01:40:08.399+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[40046]: debug 2024-02-18T01:40:08.399+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[40046]: debug 2024-02-18T01:40:08.399+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[34908]: debug 2024-02-18T01:40:08.631+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[34908]: debug 2024-02-18T01:40:08.631+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:08.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[34908]: debug 2024-02-18T01:40:08.631+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:09.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[29937]: debug 2024-02-18T01:40:08.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:23.898208+0000 front 2024-02-18T01:39:23.898475+0000 (oldest deadline 2024-02-18T01:39:49.798138+0000) 2024-02-18T01:40:09.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[29937]: debug 2024-02-18T01:40:08.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:35.099691+0000 front 2024-02-18T01:39:35.099805+0000 (oldest deadline 2024-02-18T01:39:57.399838+0000) 2024-02-18T01:40:09.243 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:08 smithi155 bash[29937]: debug 2024-02-18T01:40:08.959+0000 7f28a2e09700 -1 osd.5 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:43.301574+0000 front 2024-02-18T01:39:43.301494+0000 (oldest deadline 2024-02-18T01:40:06.801459+0000) 2024-02-18T01:40:09.663 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[40046]: debug 2024-02-18T01:40:09.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:09.663 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[40046]: debug 2024-02-18T01:40:09.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:09.663 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[40046]: debug 2024-02-18T01:40:09.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:09.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[34908]: debug 2024-02-18T01:40:09.659+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:09.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[34908]: debug 2024-02-18T01:40:09.659+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:09.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:09 smithi155 bash[34908]: debug 2024-02-18T01:40:09.659+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:10.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:10 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:10.149Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:10.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:10 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:10.149Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:10.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:10 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:10.150Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:10.496 INFO:journalctl@ceph.osd.5.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[50380]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-5 2024-02-18T01:40:10.496 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[40046]: debug 2024-02-18T01:40:10.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:10.496 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[40046]: debug 2024-02-18T01:40:10.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:10.496 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[40046]: debug 2024-02-18T01:40:10.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:10.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[34908]: debug 2024-02-18T01:40:10.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:10.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[34908]: debug 2024-02-18T01:40:10.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:10.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:10 smithi155 bash[34908]: debug 2024-02-18T01:40:10.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:11.735 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[40046]: debug 2024-02-18T01:40:11.403+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:11.735 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[40046]: debug 2024-02-18T01:40:11.403+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:11.736 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[40046]: debug 2024-02-18T01:40:11.403+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:11.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[34908]: debug 2024-02-18T01:40:11.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:11.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[34908]: debug 2024-02-18T01:40:11.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:11.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:11 smithi155 bash[34908]: debug 2024-02-18T01:40:11.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:12.458 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.5.service' 2024-02-18T01:40:12.489 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:12.489 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-02-18T01:40:12.489 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-02-18T01:40:12.489 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.6 2024-02-18T01:40:12.712 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 systemd[1]: Stopping Ceph osd.6 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:12.712 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[50636]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.6 2024-02-18T01:40:12.712 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[40046]: debug 2024-02-18T01:40:12.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:12.712 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[40046]: debug 2024-02-18T01:40:12.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:12.712 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[40046]: debug 2024-02-18T01:40:12.359+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.707+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.743+0000 7f575939c700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.743+0000 7f575939c700 -1 osd.6 746 *** Got signal Terminated *** 2024-02-18T01:40:12.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:12 smithi155 bash[34908]: debug 2024-02-18T01:40:12.743+0000 7f575939c700 -1 osd.6 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:40:13.676 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[40046]: debug 2024-02-18T01:40:13.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:13.676 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[40046]: debug 2024-02-18T01:40:13.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:13.676 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[40046]: debug 2024-02-18T01:40:13.367+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:13.676 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[40046]: debug 2024-02-18T01:40:13.367+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:13.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:13 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:13.579Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:13.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:13 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:13.580Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:13.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:13 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:13.581Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:13.992 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[34908]: debug 2024-02-18T01:40:13.671+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:13.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[34908]: debug 2024-02-18T01:40:13.671+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:13.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[34908]: debug 2024-02-18T01:40:13.671+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:13.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:13 smithi155 bash[34908]: debug 2024-02-18T01:40:13.671+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:49.993637+0000 front 2024-02-18T01:39:49.993539+0000 (oldest deadline 2024-02-18T01:40:13.493449+0000) 2024-02-18T01:40:14.686 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[40046]: debug 2024-02-18T01:40:14.327+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:14.686 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[40046]: debug 2024-02-18T01:40:14.327+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:14.686 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[40046]: debug 2024-02-18T01:40:14.327+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:14.687 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[40046]: debug 2024-02-18T01:40:14.327+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:14.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[34908]: debug 2024-02-18T01:40:14.683+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:14.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[34908]: debug 2024-02-18T01:40:14.683+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:14.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[34908]: debug 2024-02-18T01:40:14.683+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:14.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:14 smithi155 bash[34908]: debug 2024-02-18T01:40:14.683+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:49.993637+0000 front 2024-02-18T01:39:49.993539+0000 (oldest deadline 2024-02-18T01:40:13.493449+0000) 2024-02-18T01:40:15.735 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[40046]: debug 2024-02-18T01:40:15.371+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:15.735 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[40046]: debug 2024-02-18T01:40:15.371+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:15.735 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[40046]: debug 2024-02-18T01:40:15.371+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:15.736 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[40046]: debug 2024-02-18T01:40:15.371+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:15.736 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[40046]: debug 2024-02-18T01:40:15.371+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:15.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[34908]: debug 2024-02-18T01:40:15.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:15.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[34908]: debug 2024-02-18T01:40:15.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:15.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[34908]: debug 2024-02-18T01:40:15.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:15.993 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:15 smithi155 bash[34908]: debug 2024-02-18T01:40:15.731+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:49.993637+0000 front 2024-02-18T01:39:49.993539+0000 (oldest deadline 2024-02-18T01:40:13.493449+0000) 2024-02-18T01:40:16.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[40046]: debug 2024-02-18T01:40:16.335+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:16.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[40046]: debug 2024-02-18T01:40:16.335+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:16.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[40046]: debug 2024-02-18T01:40:16.335+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:16.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[40046]: debug 2024-02-18T01:40:16.335+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:16.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[40046]: debug 2024-02-18T01:40:16.335+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:17.243 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[34908]: debug 2024-02-18T01:40:16.775+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:28.087445+0000 front 2024-02-18T01:39:28.087457+0000 (oldest deadline 2024-02-18T01:39:50.387068+0000) 2024-02-18T01:40:17.243 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[34908]: debug 2024-02-18T01:40:16.775+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:32.088697+0000 front 2024-02-18T01:39:32.088749+0000 (oldest deadline 2024-02-18T01:39:56.788639+0000) 2024-02-18T01:40:17.243 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[34908]: debug 2024-02-18T01:40:16.775+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:39.089998+0000 front 2024-02-18T01:39:39.090248+0000 (oldest deadline 2024-02-18T01:40:04.989915+0000) 2024-02-18T01:40:17.243 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:16 smithi155 bash[34908]: debug 2024-02-18T01:40:16.775+0000 7f5754315700 -1 osd.6 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:49.993637+0000 front 2024-02-18T01:39:49.993539+0000 (oldest deadline 2024-02-18T01:40:13.493449+0000) 2024-02-18T01:40:17.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:17 smithi155 bash[40046]: debug 2024-02-18T01:40:17.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:17.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:17 smithi155 bash[40046]: debug 2024-02-18T01:40:17.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:17.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:17 smithi155 bash[40046]: debug 2024-02-18T01:40:17.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:17.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:17 smithi155 bash[40046]: debug 2024-02-18T01:40:17.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:17.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:17 smithi155 bash[40046]: debug 2024-02-18T01:40:17.383+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.6.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[50636]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-6 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[40046]: debug 2024-02-18T01:40:18.431+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[40046]: debug 2024-02-18T01:40:18.431+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[40046]: debug 2024-02-18T01:40:18.431+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[40046]: debug 2024-02-18T01:40:18.431+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:18.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:18 smithi155 bash[40046]: debug 2024-02-18T01:40:18.431+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:19.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:19 smithi155 bash[40046]: debug 2024-02-18T01:40:19.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:19.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:19 smithi155 bash[40046]: debug 2024-02-18T01:40:19.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:19.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:19 smithi155 bash[40046]: debug 2024-02-18T01:40:19.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:19.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:19 smithi155 bash[40046]: debug 2024-02-18T01:40:19.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:19.744 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:19 smithi155 bash[40046]: debug 2024-02-18T01:40:19.419+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:20.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:20 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:20.150Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:20.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:20 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:20.151Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:20.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:20 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:20.152Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:20.579 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.6.service' 2024-02-18T01:40:20.601 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:20.601 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-02-18T01:40:20.602 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-02-18T01:40:20.602 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.7 2024-02-18T01:40:20.656 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:20.656 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:20.656 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:20.656 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.383+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:20.656 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.383+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:20.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 systemd[1]: Stopping Ceph osd.7 for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:20.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[50898]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd.7 2024-02-18T01:40:20.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.855+0000 7f12f9363700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:40:20.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.855+0000 7f12f9363700 -1 osd.7 746 *** Got signal Terminated *** 2024-02-18T01:40:20.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:20 smithi155 bash[40046]: debug 2024-02-18T01:40:20.855+0000 7f12f9363700 -1 osd.7 746 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-02-18T01:40:21.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:21 smithi155 bash[40046]: debug 2024-02-18T01:40:21.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:21.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:21 smithi155 bash[40046]: debug 2024-02-18T01:40:21.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:21.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:21 smithi155 bash[40046]: debug 2024-02-18T01:40:21.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:21.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:21 smithi155 bash[40046]: debug 2024-02-18T01:40:21.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:21.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:21 smithi155 bash[40046]: debug 2024-02-18T01:40:21.355+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:22.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:22 smithi155 bash[40046]: debug 2024-02-18T01:40:22.379+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:22.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:22 smithi155 bash[40046]: debug 2024-02-18T01:40:22.379+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:22.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:22 smithi155 bash[40046]: debug 2024-02-18T01:40:22.379+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:22.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:22 smithi155 bash[40046]: debug 2024-02-18T01:40:22.379+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:22.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:22 smithi155 bash[40046]: debug 2024-02-18T01:40:22.379+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:23.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:23 smithi155 bash[40046]: debug 2024-02-18T01:40:23.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:23.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:23 smithi155 bash[40046]: debug 2024-02-18T01:40:23.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:23.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:23 smithi155 bash[40046]: debug 2024-02-18T01:40:23.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:23.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:23 smithi155 bash[40046]: debug 2024-02-18T01:40:23.419+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:23.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:23 smithi155 bash[40046]: debug 2024-02-18T01:40:23.419+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:23.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:23 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:23.579Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:23.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:23 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:23.580Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:23.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:23 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:23.581Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:24.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:24 smithi155 bash[40046]: debug 2024-02-18T01:40:24.391+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:24.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:24 smithi155 bash[40046]: debug 2024-02-18T01:40:24.391+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:24.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:24 smithi155 bash[40046]: debug 2024-02-18T01:40:24.391+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:24.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:24 smithi155 bash[40046]: debug 2024-02-18T01:40:24.391+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:24.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:24 smithi155 bash[40046]: debug 2024-02-18T01:40:24.391+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6803 osd.0 since back 2024-02-18T01:39:26.681450+0000 front 2024-02-18T01:39:26.681500+0000 (oldest deadline 2024-02-18T01:39:48.981529+0000) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6807 osd.1 since back 2024-02-18T01:39:34.183742+0000 front 2024-02-18T01:39:34.183827+0000 (oldest deadline 2024-02-18T01:39:59.483564+0000) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6811 osd.2 since back 2024-02-18T01:39:42.886019+0000 front 2024-02-18T01:39:42.885748+0000 (oldest deadline 2024-02-18T01:40:08.185658+0000) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.105:6815 osd.3 since back 2024-02-18T01:39:51.588131+0000 front 2024-02-18T01:39:51.588169+0000 (oldest deadline 2024-02-18T01:40:14.488134+0000) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 heartbeat_check: no reply from 172.21.15.155:6802 osd.4 since back 2024-02-18T01:40:00.191114+0000 front 2024-02-18T01:40:00.191044+0000 (oldest deadline 2024-02-18T01:40:24.890930+0000) 2024-02-18T01:40:25.743 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:25 smithi155 bash[40046]: debug 2024-02-18T01:40:25.355+0000 7f12f22d8700 -1 osd.7 746 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24452.0:1337 2.0 2:353ec8f9:::gateway.conf:head [getxattr epoch in=5b] snapc 0=[] ondisk+read+known_if_redirected e746) 2024-02-18T01:40:26.993 INFO:journalctl@ceph.osd.7.smithi155.stdout:Feb 18 01:40:26 smithi155 bash[50898]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-osd-7 2024-02-18T01:40:28.521 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@osd.7.service' 2024-02-18T01:40:28.592 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:28.593 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-02-18T01:40:28.593 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-02-18T01:40:28.593 DEBUG:teuthology.orchestra.run.smithi105:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@rgw.foo.a 2024-02-18T01:40:28.989 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:Feb 18 01:40:28 smithi105 systemd[1]: Stopping Ceph rgw.foo.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:28.989 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:Feb 18 01:40:28 smithi105 bash[130462]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-rgw.foo.a 2024-02-18T01:40:28.989 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:Feb 18 01:40:28 smithi105 bash[49375]: debug 2024-02-18T01:40:28.796+0000 7f0c893f9700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-02-18T01:40:28.989 INFO:journalctl@ceph.rgw.foo.a.smithi105.stdout:Feb 18 01:40:28 smithi105 bash[49375]: debug 2024-02-18T01:40:28.796+0000 7f0ca04c3880 -1 shutting down 2024-02-18T01:40:30.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:30 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:30.151Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:30.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:30 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:30.152Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:30.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:30 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:30.153Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:33 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:33.580Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:33 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:33.581Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:33.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:33 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:33.582Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:39.627 DEBUG:teuthology.orchestra.run.smithi105:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@rgw.foo.a.service' 2024-02-18T01:40:39.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:39.649 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-02-18T01:40:39.649 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-02-18T01:40:39.649 DEBUG:teuthology.orchestra.run.smithi155:> sudo systemctl stop ceph-358e970a-cdfa-11ee-95ba-87774f69a715@prometheus.a 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 systemd[1]: Stopping Ceph prometheus.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[51163]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-prometheus.a 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:775 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:798 level=info msg="Stopping scrape discovery manager..." 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:812 level=info msg="Stopping notify discovery manager..." 2024-02-18T01:40:39.938 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:834 level=info msg="Stopping scrape manager..." 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:808 level=info msg="Notify discovery manager stopped" 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:794 level=info msg="Scrape discovery manager stopped" 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=manager.go:945 level=info component="rule manager" msg="Stopping rule manager..." 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=main.go:828 level=info msg="Scrape manager stopped" 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.882Z caller=manager.go:955 level=info component="rule manager" msg="Rule manager stopped" 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.884Z caller=notifier.go:600 level=info component=notifier msg="Stopping notification manager..." 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.884Z caller=main.go:1054 level=info msg="Notifier manager stopped" 2024-02-18T01:40:39.939 INFO:journalctl@ceph.prometheus.a.smithi155.stdout:Feb 18 01:40:39 smithi155 bash[47821]: ts=2024-02-18T01:40:39.884Z caller=main.go:1066 level=info msg="See you next time!" 2024-02-18T01:40:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:40 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:40.152Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:40 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:40.153Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:40.489 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:40 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:40.154Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:40.596 DEBUG:teuthology.orchestra.run.smithi155:> sudo pkill -f 'journalctl -f -n 0 -u ceph-358e970a-cdfa-11ee-95ba-87774f69a715@prometheus.a.service' 2024-02-18T01:40:40.623 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-02-18T01:40:40.623 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-02-18T01:40:40.623 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm rm-cluster --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T01:40:41.991 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:41 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:41.628Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:41.991 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:41 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:41.630Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:43.988 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:43 smithi105 bash[54021]: level=error ts=2024-02-18T01:40:43.581Z caller=dispatch.go:354 component=dispatcher msg="Notify for alerts failed" num_alerts=3 err="ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:43.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:43 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:43.582Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi105.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.105:8443: connect: connection refused" 2024-02-18T01:40:43.989 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:43 smithi105 bash[54021]: level=warn ts=2024-02-18T01:40:43.583Z caller=notify.go:724 component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi155.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.155:8443: connect: connection refused" 2024-02-18T01:40:46.712 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:46 smithi105 systemd[1]: Stopping Ceph node-exporter.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:46.712 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:46 smithi105 bash[130861]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-node-exporter.a 2024-02-18T01:40:47.694 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:47 smithi105 bash[130861]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-node-exporter-a 2024-02-18T01:40:47.695 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:47 smithi105 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-02-18T01:40:47.695 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:47 smithi105 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-02-18T01:40:47.695 INFO:journalctl@ceph.node-exporter.a.smithi105.stdout:Feb 18 01:40:47 smithi105 systemd[1]: Stopped Ceph node-exporter.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:40:47.695 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:47 smithi105 systemd[1]: Stopping Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:47.695 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:47 smithi105 bash[131008]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-alertmanager.a 2024-02-18T01:40:47.988 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:47 smithi105 bash[54021]: level=info ts=2024-02-18T01:40:47.694Z caller=main.go:557 msg="Received SIGTERM, exiting gracefully..." 2024-02-18T01:40:48.692 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:48 smithi105 bash[131008]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-alertmanager-a 2024-02-18T01:40:48.692 INFO:journalctl@ceph.alertmanager.a.smithi105.stdout:Feb 18 01:40:48 smithi105 systemd[1]: Stopped Ceph alertmanager.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:40:48.898 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm rm-cluster --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 --force --keep-logs 2024-02-18T01:40:54.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:40:54 smithi155 systemd[1]: Stopping Ceph iscsi.iscsi.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:40:54.640 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:40:54 smithi155 bash[51506]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-iscsi.iscsi.a 2024-02-18T01:40:54.992 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:40:54 smithi155 bash[43865]: debug Shutdown received 2024-02-18T01:41:05.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:41:05 smithi155 bash[51506]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-iscsi-iscsi-a 2024-02-18T01:41:05.743 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:41:05 smithi155 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-02-18T01:41:06.407 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:06 smithi155 systemd[1]: Stopping Ceph node-exporter.b for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:41:06.407 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:06 smithi155 bash[51708]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-node-exporter.b 2024-02-18T01:41:06.407 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:41:06 smithi155 bash[51597]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-iscsi-iscsi-a-tcmu 2024-02-18T01:41:06.408 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:41:06 smithi155 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-02-18T01:41:06.408 INFO:journalctl@ceph.iscsi.iscsi.a.smithi155.stdout:Feb 18 01:41:06 smithi155 systemd[1]: Stopped Ceph iscsi.iscsi.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:41:07.243 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:06 smithi155 bash[51708]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-node-exporter-b 2024-02-18T01:41:07.243 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:06 smithi155 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-02-18T01:41:07.243 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:07 smithi155 systemd[1]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-02-18T01:41:07.243 INFO:journalctl@ceph.node-exporter.b.smithi155.stdout:Feb 18 01:41:07 smithi155 systemd[1]: Stopped Ceph node-exporter.b for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:41:07.547 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:41:07 smithi155 systemd[1]: Stopping Ceph grafana.a for 358e970a-cdfa-11ee-95ba-87774f69a715... 2024-02-18T01:41:07.547 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:41:07 smithi155 bash[51889]: Error response from daemon: No such container: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-grafana.a 2024-02-18T01:41:07.547 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:41:07 smithi155 bash[46955]: t=2024-02-18T01:41:07+0000 lvl=info msg="Shutdown started" logger=server reason="System signal: terminated" 2024-02-18T01:41:08.455 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:41:08 smithi155 bash[51889]: ceph-358e970a-cdfa-11ee-95ba-87774f69a715-grafana-a 2024-02-18T01:41:08.455 INFO:journalctl@ceph.grafana.a.smithi155.stdout:Feb 18 01:41:08 smithi155 systemd[1]: Stopped Ceph grafana.a for 358e970a-cdfa-11ee-95ba-87774f69a715. 2024-02-18T01:41:08.645 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:41:08.660 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-02-18T01:41:08.675 INFO:tasks.cephadm:Archiving crash dumps... 2024-02-18T01:41:08.677 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/ubuntu@smithi105.front.sepia.ceph.com/crash 2024-02-18T01:41:08.678 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash -- . 2024-02-18T01:41:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:tar: /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T01:41:08.718 INFO:teuthology.orchestra.run.smithi105.stderr:tar: Error is not recoverable: exiting now 2024-02-18T01:41:08.720 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/ubuntu@smithi155.front.sepia.ceph.com/crash 2024-02-18T01:41:08.721 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash -- . 2024-02-18T01:41:08.736 INFO:teuthology.orchestra.run.smithi155.stderr:tar: /var/lib/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/crash: Cannot open: No such file or directory 2024-02-18T01:41:08.736 INFO:teuthology.orchestra.run.smithi155.stderr:tar: Error is not recoverable: exiting now 2024-02-18T01:41:08.737 INFO:tasks.cephadm:Checking cluster log for badness... 2024-02-18T01:41:08.737 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'pool application' | egrep -v 'mons down' | egrep -v 'out of quorum' | egrep -v 'needs hit_set_type to be set but it is not' | head -n 1 2024-02-18T01:41:08.783 INFO:teuthology.orchestra.run.smithi105.stdout:2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:41:08.783 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-02-18T01:41:08.783 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[SEC\]' /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'pool application' | egrep -v 'mons down' | egrep -v 'out of quorum' | egrep -v 'needs hit_set_type to be set but it is not' | head -n 1 2024-02-18T01:41:08.844 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[ERR\]' /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'pool application' | egrep -v 'mons down' | egrep -v 'out of quorum' | egrep -v 'needs hit_set_type to be set but it is not' | head -n 1 2024-02-18T01:41:08.908 DEBUG:teuthology.orchestra.run.smithi105:> sudo egrep '\[WRN\]' /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(MON_DOWN\)' | egrep -v 'missing hit_sets' | egrep -v 'do not have an application enabled' | egrep -v 'application not enabled on pool' | egrep -v 'pool application' | egrep -v 'mons down' | egrep -v 'out of quorum' | egrep -v 'needs hit_set_type to be set but it is not' | head -n 1 2024-02-18T01:41:08.967 INFO:teuthology.orchestra.run.smithi105.stdout:2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-02-18T01:41:08.968 INFO:tasks.cephadm:Compressing logs... 2024-02-18T01:41:08.968 DEBUG:teuthology.orchestra.run.smithi105:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-18T01:41:09.012 DEBUG:teuthology.orchestra.run.smithi155:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-02-18T01:41:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-18T01:41:09.024 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-18T01:41:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-18T01:41:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.0.log 2024-02-18T01:41:09.025 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.a.log 2024-02-18T01:41:09.026 INFO:teuthology.orchestra.run.smithi105.stderr:gzip /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.0.log: -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-18T01:41:09.027 INFO:teuthology.orchestra.run.smithi155.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-02-18T01:41:09.027 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-02-18T01:41:09.028 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.y.log 2024-02-18T01:41:09.029 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log: 89.1% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-18T01:41:09.029 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.1.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.6.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.b.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/tcmu-runner.log 2024-02-18T01:41:09.031 INFO:teuthology.orchestra.run.smithi155.stderr: 81.6% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.cephadm.log.gz 2024-02-18T01:41:09.032 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.7.log 2024-02-18T01:41:09.032 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/tcmu-runner.log: 68.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/tcmu-runner.log.gz 2024-02-18T01:41:09.032 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-client.rgw.foo.a.log 2024-02-18T01:41:09.032 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.5.log 2024-02-18T01:41:09.035 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.1.log: /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-client.rgw.foo.a.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-18T01:41:09.035 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.2.log 2024-02-18T01:41:09.035 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log 2024-02-18T01:41:09.035 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.7.log: /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.x.log 2024-02-18T01:41:09.036 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.4.log 2024-02-18T01:41:09.038 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log: 91.2% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-18T01:41:09.039 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.3.log 2024-02-18T01:41:09.039 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.x.log: 88.7% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-18T01:41:09.039 INFO:teuthology.orchestra.run.smithi155.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log 2024-02-18T01:41:09.045 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.4.log: 91.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-volume.log.gz 2024-02-18T01:41:09.046 INFO:teuthology.orchestra.run.smithi155.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log: 92.3% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-18T01:41:09.046 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.2.log: 90.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-02-18T01:41:09.046 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log 2024-02-18T01:41:09.047 INFO:teuthology.orchestra.run.smithi155.stderr: 92.1% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.x.log.gz 2024-02-18T01:41:09.050 INFO:teuthology.orchestra.run.smithi155.stderr: 87.6% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-18T01:41:09.057 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.3.log: 95.2% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.audit.log.gz 2024-02-18T01:41:09.057 INFO:teuthology.orchestra.run.smithi105.stderr:gzip -5 --verbose -- /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.c.log 2024-02-18T01:41:09.064 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log: 93.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph.log.gz 2024-02-18T01:41:09.088 INFO:teuthology.orchestra.run.smithi105.stderr:/var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.c.log: 92.8% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-02-18T01:41:09.450 INFO:teuthology.orchestra.run.smithi105.stderr: 90.4% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mgr.y.log.gz 2024-02-18T01:41:10.011 INFO:teuthology.orchestra.run.smithi105.stderr: 92.7% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.c.log.gz 2024-02-18T01:41:10.570 INFO:teuthology.orchestra.run.smithi155.stderr: 92.7% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.b.log.gz 2024-02-18T01:41:11.041 INFO:teuthology.orchestra.run.smithi105.stderr: 91.7% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-mon.a.log.gz 2024-02-18T01:41:13.394 INFO:teuthology.orchestra.run.smithi155.stderr: 93.9% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.6.log.gz 2024-02-18T01:41:13.512 INFO:teuthology.orchestra.run.smithi105.stderr: 94.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.2.log.gz 2024-02-18T01:41:13.550 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.5.log.gz 2024-02-18T01:41:13.752 INFO:teuthology.orchestra.run.smithi155.stderr: 94.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.7.log.gz 2024-02-18T01:41:14.250 INFO:teuthology.orchestra.run.smithi105.stderr: 94.0% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.0.log.gz 2024-02-18T01:41:14.358 INFO:teuthology.orchestra.run.smithi155.stderr: 94.1% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.4.log.gz 2024-02-18T01:41:14.358 INFO:teuthology.orchestra.run.smithi155.stderr: 2024-02-18T01:41:14.358 INFO:teuthology.orchestra.run.smithi155.stderr:real 0m5.342s 2024-02-18T01:41:14.359 INFO:teuthology.orchestra.run.smithi155.stderr:user 0m19.836s 2024-02-18T01:41:14.359 INFO:teuthology.orchestra.run.smithi155.stderr:sys 0m0.707s 2024-02-18T01:41:14.560 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.1.log.gz 2024-02-18T01:41:14.724 INFO:teuthology.orchestra.run.smithi105.stderr: 94.1% -- replaced with /var/log/ceph/358e970a-cdfa-11ee-95ba-87774f69a715/ceph-osd.3.log.gz 2024-02-18T01:41:14.725 INFO:teuthology.orchestra.run.smithi105.stderr: 2024-02-18T01:41:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:real 0m5.712s 2024-02-18T01:41:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:user 0m23.457s 2024-02-18T01:41:14.725 INFO:teuthology.orchestra.run.smithi105.stderr:sys 0m0.952s 2024-02-18T01:41:14.725 INFO:tasks.cephadm:Archiving logs... 2024-02-18T01:41:14.726 DEBUG:teuthology.misc:Transferring archived files from smithi105:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/ubuntu@smithi105.front.sepia.ceph.com/log 2024-02-18T01:41:14.727 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-18T01:41:18.096 DEBUG:teuthology.misc:Transferring archived files from smithi155:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/ubuntu@smithi155.front.sepia.ceph.com/log 2024-02-18T01:41:18.097 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /var/log/ceph -- . 2024-02-18T01:41:21.044 INFO:tasks.cephadm:Removing cluster... 2024-02-18T01:41:21.045 DEBUG:teuthology.orchestra.run.smithi105:> sudo cephadm rm-cluster --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 --force 2024-02-18T01:41:22.567 DEBUG:teuthology.orchestra.run.smithi155:> sudo cephadm rm-cluster --fsid 358e970a-cdfa-11ee-95ba-87774f69a715 --force 2024-02-18T01:41:24.097 INFO:tasks.cephadm:Teardown complete 2024-02-18T01:41:24.098 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-02-18T01:41:24.109 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-02-18T01:41:24.109 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T01:41:24.111 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-02-18T01:41:24.185 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-02-18T01:41:24.194 INFO:teuthology.task.clock:Checking final clock skew... 2024-02-18T01:41:24.195 DEBUG:teuthology.orchestra.run.smithi105:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T01:41:24.198 DEBUG:teuthology.orchestra.run.smithi155:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-02-18T01:41:24.218 INFO:teuthology.orchestra.run.smithi155.stdout: remote refid st t when poll reach delay offset jitter 2024-02-18T01:41:24.218 INFO:teuthology.orchestra.run.smithi155.stdout:============================================================================== 2024-02-18T01:41:24.218 INFO:teuthology.orchestra.run.smithi155.stdout:*hv01.front.sepi 44.4.53.2 2 u 131 128 377 0.079 -1.266 0.449 2024-02-18T01:41:24.218 INFO:teuthology.orchestra.run.smithi155.stdout:+hv02.front.sepi 63.231.80.2 3 u 73 128 377 0.122 2.510 0.253 2024-02-18T01:41:24.219 INFO:teuthology.orchestra.run.smithi155.stdout:+hv03.front.sepi 74.6.168.72 3 u 63 128 377 0.130 1.753 0.240 2024-02-18T01:41:24.219 INFO:teuthology.orchestra.run.smithi155.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout: remote refid st t when poll reach delay offset jitter 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout:============================================================================== 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout:*hv01.front.sepi 44.4.53.2 2 u 3 64 377 0.088 -1.375 0.126 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout:+hv02.front.sepi 63.231.80.2 3 u 1 64 377 0.095 2.840 0.111 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout:+hv03.front.sepi 74.6.168.72 3 u 63 64 377 0.090 2.261 0.152 2024-02-18T01:41:24.220 INFO:teuthology.orchestra.run.smithi105.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 0.000 0.000 2024-02-18T01:41:24.221 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-02-18T01:41:24.230 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-02-18T01:41:24.232 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-02-18T01:41:24.240 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-02-18T01:41:24.248 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-02-18T01:41:24.257 INFO:teuthology.task.internal:Duration was 2430.727834 seconds 2024-02-18T01:41:24.257 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-02-18T01:41:24.267 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-02-18T01:41:24.267 DEBUG:teuthology.orchestra.run.smithi105:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T01:41:24.269 DEBUG:teuthology.orchestra.run.smithi155:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-02-18T01:41:24.317 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-02-18T01:41:24.318 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi105.front.sepia.ceph.com 2024-02-18T01:41:24.318 DEBUG:teuthology.orchestra.run.smithi105:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T01:41:24.334 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi155.front.sepia.ceph.com 2024-02-18T01:41:24.334 DEBUG:teuthology.orchestra.run.smithi155:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-02-18T01:41:24.372 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-02-18T01:41:24.373 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T01:41:24.377 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-02-18T01:41:24.481 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-02-18T01:41:24.481 DEBUG:teuthology.orchestra.run.smithi105:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T01:41:24.483 DEBUG:teuthology.orchestra.run.smithi155:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-02-18T01:41:24.628 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-02-18T01:41:24.638 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-02-18T01:41:24.639 DEBUG:teuthology.orchestra.run.smithi105:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T01:41:24.651 DEBUG:teuthology.orchestra.run.smithi155:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-02-18T01:41:24.666 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-02-18T01:41:24.677 DEBUG:teuthology.orchestra.run.smithi105:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T01:41:24.693 DEBUG:teuthology.orchestra.run.smithi155:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-02-18T01:41:24.705 INFO:teuthology.orchestra.run.smithi105.stdout:kernel.core_pattern = core 2024-02-18T01:41:24.721 INFO:teuthology.orchestra.run.smithi155.stdout:kernel.core_pattern = core 2024-02-18T01:41:24.747 DEBUG:teuthology.orchestra.run.smithi105:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T01:41:24.768 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T01:41:24.769 DEBUG:teuthology.orchestra.run.smithi155:> test -e /home/ubuntu/cephtest/archive/coredump 2024-02-18T01:41:24.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-02-18T01:41:24.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-02-18T01:41:24.803 INFO:teuthology.task.internal:Transferring archived files... 2024-02-18T01:41:24.804 DEBUG:teuthology.misc:Transferring archived files from smithi105:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/smithi105 2024-02-18T01:41:24.805 DEBUG:teuthology.orchestra.run.smithi105:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T01:41:24.845 DEBUG:teuthology.misc:Transferring archived files from smithi155:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039/remote/smithi155 2024-02-18T01:41:24.845 DEBUG:teuthology.orchestra.run.smithi155:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-02-18T01:41:24.876 INFO:teuthology.task.internal:Removing archive directory... 2024-02-18T01:41:24.876 DEBUG:teuthology.orchestra.run.smithi105:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T01:41:24.881 DEBUG:teuthology.orchestra.run.smithi155:> rm -rf -- /home/ubuntu/cephtest/archive 2024-02-18T01:41:24.919 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-02-18T01:41:24.929 INFO:teuthology.task.internal:Not uploading archives. 2024-02-18T01:41:24.929 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-02-18T01:41:24.939 INFO:teuthology.task.internal:Tidying up after the test... 2024-02-18T01:41:24.939 DEBUG:teuthology.orchestra.run.smithi105:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T01:41:24.941 DEBUG:teuthology.orchestra.run.smithi155:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-02-18T01:41:24.948 INFO:teuthology.orchestra.run.smithi105.stdout: 659550 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 01:41 /home/ubuntu/cephtest 2024-02-18T01:41:24.968 INFO:teuthology.orchestra.run.smithi155.stdout: 659551 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 18 01:41 /home/ubuntu/cephtest 2024-02-18T01:41:24.970 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-02-18T01:41:24.981 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-02-18T01:41:25.038 INFO:teuthology.nuke:Checking targets against current locks 2024-02-18T01:41:25.066 DEBUG:teuthology.nuke:shortname: smithi105 2024-02-18T01:41:25.066 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T01:41:25.088 DEBUG:teuthology.nuke:shortname: smithi155 2024-02-18T01:41:25.088 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-02-18T01:41:25.093 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi105.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:52:23.760472', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE='} 2024-02-18T01:41:25.116 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi155.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-02-17_15:55:51-rados-pacific-release-distro-default-smithi/7565039', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '18.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-02-18 00:52:23.758881', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHSAlWjESMc0Gra5f/6dvvbjuYJc2qVK13Ztw1gt4S0hdg9pLl/eAcLsaHQz3OAGEzcbKS0DF4bknjuwe5HHhPE='} 2024-02-18T01:41:25.142 INFO:teuthology.orchestra.console.smithi105:Power off 2024-02-18T01:41:25.143 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T01:41:25.171 DEBUG:teuthology.orchestra.console.smithi105:power off output: Chassis Power Control: Down/Off 2024-02-18T01:41:25.171 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:25.191 INFO:teuthology.orchestra.console.smithi155:Power off 2024-02-18T01:41:25.191 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-02-18T01:41:25.203 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-02-18T01:41:25.217 DEBUG:teuthology.orchestra.console.smithi155:power off output: Chassis Power Control: Down/Off 2024-02-18T01:41:25.217 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:25.244 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-18T01:41:29.204 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:29.248 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:29.317 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-02-18T01:41:29.358 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-18T01:41:33.320 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:33.359 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:33.432 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is on 2024-02-18T01:41:33.469 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is on 2024-02-18T01:41:37.436 DEBUG:teuthology.orchestra.console.smithi105:pexpect command: ipmitool -H smithi105.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:37.470 DEBUG:teuthology.orchestra.console.smithi155:pexpect command: ipmitool -H smithi155.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-02-18T01:41:37.546 DEBUG:teuthology.orchestra.console.smithi105:check power output: Chassis Power is off 2024-02-18T01:41:37.579 DEBUG:teuthology.orchestra.console.smithi155:check power output: Chassis Power is off 2024-02-18T01:41:37.648 INFO:teuthology.orchestra.console.smithi105:Power off completed 2024-02-18T01:41:37.680 INFO:teuthology.orchestra.console.smithi155:Power off completed 2024-02-18T01:41:37.785 INFO:teuthology.run:Summary data: description: rados/cephadm/with-work/{0-distro/ubuntu_18.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} duration: 2430.727834224701 failure_reason: '"2024-02-18T01:23:11.389015+0000 mon.a (mon.0) 769 : cluster [WRN] Health check failed: 17 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2024-02-18T01:41:37.785 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-02-18T01:41:37.866 INFO:teuthology.run:FAIL